benchmarks (was: Self 4.1 for Mac OS Its too slow to be used for anything useful.

jecel at lsi.usp.br jecel at lsi.usp.br
Fri Nov 5 21:27:16 UTC 1999


I have run a few simple benchmarks to compare Squeak and Self on a Sparc
machine. I use a Linux PC for Squeak, so the version I have on the Sparc
is far from the latest. If that makes much of a difference in these short
codes, let me know and I'll try again with Squeak 2.6.

I'll list all relevant sources here so people can easily see if I have
made any mistakes.

-------------------------------
Squeak 2.2 on an Ultra 5 Sun Sparcstation

this code:

| r t | t _ Time millisecondsToRun: [r _ 26 benchFib].
        r//t*1000

generated these results when executed five times:

     397000  392000  402000  394000  389000

where:

benchFib
  ^ self < 2
        ifTrue: [1]
        False: [(self-1) benchFib + (self-2) benchFib + 1]

this code:

5000000 // (Time millisecondsToRun: [10 benchmark]) * 1000


generated these results when executed five times:

     10121000  10020000  9920000  10121000  10101000

where:           

benchmark                                        
  | size flags prime k count |
  size _ 8190.                             
  1 to: self do:
     [:iter |
     count _ 0.
     flags _ Array new: size atAllPut: true.
     1 to: size do:
           [:i| (flags at: i) ifTrue:
              [prime _ i+1.                               
              k _ i + prime.
              [k <= size] whileTrue:
                   [flags at: k put: false.
                   k _ k + prime].                        
               count _ count + 1]]].
   ^ count
--------------------------------------
Self 4.1 on an Ultra 5 Sun Sparcstation

this code:

| r. t | t: [ r: 26 benchFib ] realTime.
   (r/~t)*1000

generated these results when executed five times:

  14549000  15109000  4910000  17080000  16368000

where:

benchFib = (
   < 2 ifTrue: [1]
       False: [( - 1) benchFib + ( - 2) benchFib + 1]
   )

this code:
   
(5000000 /~ [10 benchmark] realTime) * 1000

generated these results when executed five times:

  31250000  68493000  46729000  70423000  73590000

where:        

benchmark = (| count. flags. k. prime. size |    
   size: 8190.
   1 to: self Do:                                
      [|:iter|
       count: 0.
       flags: vector copySize: size+1 FillingWith: true.
       1 to: size Do:
           [| :i | (flags at: i) ifTrue:
                   [prime: i+1.                      
                   k: i + prime.
                   [k <= size] whileTrue:
                      [flags at: k Put: false.
                      k: k + prime].
                   count: count + 1]]].
    count
)

-------------------------------------------------
Geometric means:

It is interesting that I wasn't able to calculate these averages in
Python (integer overflow, even with all values divided by 1000) nor
in Self (30 bit floating points don't have enough precision), so only
Squeak was up to the job:

Squeak 2.2 benchFib:    394775
Self 4.1 benchFib:    12471732

Now this was way too much of a difference, so I checked the results of
the "26 benchFib" expression and got the same results in both languages.
Then I looked at the raw computation times in both systems and got values
of around 1 second in Squeak and about 25 milliseconds in Self, which
would confirm the first results. But there is probably something I am
doing wrong here that I am not seeing.

Squeak 2.2 benchmark: 10056298
Self 4.1 benchmark:   55485842

This is much more like it - a five to one difference in speed is what
I normally see between Self and Squeak for large applications. Note
that there are situation in which Squeak actually feels faster, and
you should never read much into benchmarks (specially ones such as
these) anyway.

Cheers,
-- Jecel





More information about the Squeak-dev mailing list