[Seaside] Benchmarks
Dominic Letz
dominic.letz at berlin.de
Sat Feb 3 00:58:24 UTC 2007
Hello everyone,
inspired by the recent Seaside benchmark discussion, I played around
benchmarking. My quest was to get the fastest "Hello World" HTTP-Answer
out of Squeak. For the benchmark I used ApacheBench 2.0.40 with the
command line "ab -n 1000 [url]". This means 1000 consecutive http request
for "Hello World".
I tried three combinations (Everything in Windows on the same machine,
setup details at the end in the ApacheBench logfiles)
1. Apache + PHP (XAMPP, no php bytecode cache - hence reparsing at every
request)
2. Squeak + Commanche
3. Squeak + JustTCP Answer
In every of these cases the html document has the same size and content of
24 bytes. However the http header size differs in each case (server
signature). Before starting the tests i expected parsing+executing an php
document would be much slower (in terms of requests served per second)
than squeak smalltalk execution. Short results in requests per second:
1. Apache + PHP 536.86
2. Squeak + Commanche 499.28
3. Squeak + JustTCP Answer 1062.30
So this draws another picture. The ratio between Apache and Commanche is
nearly 1/1 and the JustTCP (implementation below) answer was *only* as
double as fast. This got me curious about where squeak does spend its
time. So I started a MessageTally in JustTCP (implementation also below).
The resulting MessageTally gave me more questions than answers => More
than fifty percent of all the time is spent in Socket>>acceptFrom: during
creation of accept sockets. Another thirty percent goes to socket closing.
I'm not into this but I think this is to much time. Wouldn't it be
possible to create a socket pool which reuses reusable parts of the
sockets (for example the registered Semaphores)?
Thats all for today.
- 1022 tallies, 1024 msec.
**Tree**
56.8% {582ms} Socket>>waitForAcceptFor:
|55.4% {567ms} Socket>>accept
| 55.3% {566ms} Socket class>>acceptFrom:
| 54.0% {553ms} Socket>>acceptFrom:
| 23.1% {237ms} primitives
| 17.9% {183ms} SystemDictionary>>registerExternalObject:
| |17.7% {181ms} ExternalSemaphoreTable
class>>registerExternalObject:
| | 11.1% {114ms} Semaphore>>critical:
| | |9.8% {100ms} primitives
| | 6.6% {68ms} ExternalSemaphoreTable
class>>safelyRegisterExternalObject:
| 8.8% {90ms} Socket>>register
| |8.8% {90ms} Socket class>>register:
| | 8.7% {89ms} WeakRegistry>>add:
| | 6.3% {65ms} WeakRegistry>>protected:
| | 6.1% {62ms} Semaphore>>critical:ifError:
| 4.1% {42ms} Semaphore class>>new
| 4.0% {41ms} Semaphore>>initSignals
43.1% {441ms} HTTPHelloWorld>>handleClient:
36.1% {370ms} Socket>>close
4.4% {45ms} Socket>>sendData:
|4.1% {42ms} primitives
2.4% {25ms} Socket>>receiveDataInto:
2.3% {24ms} Socket>>receiveDataInto:startingAt:
2.2% {23ms} primitives
**Leaves**
36.1% {370ms} Socket>>close
23.1% {237ms} Socket>>acceptFrom:
9.8% {100ms} Semaphore>>critical:
6.6% {68ms} ExternalSemaphoreTable class>>safelyRegisterExternalObject:
6.1% {62ms} Semaphore>>critical:ifError:
4.1% {42ms} Socket>>sendData:
4.0% {41ms} Semaphore>>initSignals
2.2% {23ms} Socket>>receiveDataInto:startingAt:
**Memory**
old +58,476 bytes
young +90,832 bytes
used +149,308 bytes
free -149,308 bytes
**GCs**
full 0 totalling 0ms (0.0% uptime)
incr 27 totalling 12ms (1.0% uptime), avg 0.0ms
tenures 1 (avg 27 GCs/tenure)
root table 0 overflows
Apache PHP File:
<?
echo "<html>";
echo "Hello World";
echo "</html>"
?>
Commanche Request answer: (inspect it)
(HttpService on: 8080 named: 'Example Http Service')
onRequestDo: [ :httpRequest | HttpResponse fromString: '<hmtl>Hello
World</html>'];
start.
JustTCP class (here the methods - start with inspecting "JustTCP new
start")
initialize
| crlf |
crlf := Character cr asString, Character lf asString.
buffer := ByteArray new: 1000.
answer := 'HTTP/1.1 200 OK', crlf,
'Date: Fri, 02 Feb 2007 23:25:43 GMT', crlf,
'Server: JustTCP (Win32)', crlf,
'Connection: close', crlf,
'Content-type: text/html', crlf,
'Content-length: 24', crlf, crlf,
'<html>Hello World</html>'
start
socket := Socket new.
socket listenOn: 8100 backlogSize: 10 interface: (ByteArray new: 4).
1 to: 1000 do: [:index | count := index.
self handleClient: (socket waitForAcceptFor: 1000).
].
socket close.
startTally
"self new startTally"
socket := Socket new.
socket listenOn: 8100 backlogSize: 10 interface: (ByteArray new: 4).
self handleClient: (socket waitForAcceptFor: 1000).
MessageTally spyOn: [
1 to: 999 do: [:index | count := index.
self handleClient: (socket waitForAcceptFor: 1000).
].
].
socket close.
handleClient: aSocket
aSocket receiveDataInto: buffer.
aSocket sendData: answer.
aSocket close.
============================
====APACHE + PHP RESULTS====
============================
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 1997-2005 The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Server Software: Apache/2.2.0
Server Hostname: localhost
Server Port: 80
Document Path: /bench/
Document Length: 24 bytes
Concurrency Level: 1
Time taken for tests: 1.862678 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 279000 bytes
HTML transferred: 24000 bytes
Requests per second: 536.86 [#/sec] (mean)
Time per request: 1.863 [ms] (mean)
Time per request: 1.863 [ms] (mean, across all concurrent requests)
Transfer rate: 146.03 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.3 0 10
Processing: 0 1 3.7 0 20
Waiting: 0 1 3.6 0 20
Total: 0 1 3.9 0 20
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 10
95% 10
98% 10
99% 10
100% 20 (longest request)
==========================
====SQUEAK + COMMANCHE====
==========================
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 1997-2005 The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Server Software: Comanche/6.2
Server Hostname: localhost
Server Port: 8080
Document Path: /
Document Length: 24 bytes
Concurrency Level: 1
Time taken for tests: 2.2880 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 175000 bytes
HTML transferred: 24000 bytes
Requests per second: 499.28 [#/sec] (mean)
Time per request: 2.003 [ms] (mean)
Time per request: 2.003 [ms] (mean, across all concurrent requests)
Transfer rate: 84.88 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.3 0 10
Processing: 0 1 3.9 0 20
Waiting: 0 1 3.6 0 20
Total: 0 1 4.1 0 20
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 10
95% 10
98% 10
99% 10
100% 20 (longest request)
========================
====SQUEAK + JUSTTCP====
========================
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 1997-2005 The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Server Software: JustTCP
Server Hostname: localhost
Server Port: 8100
Document Path: /
Document Length: 24 bytes
Concurrency Level: 1
Time taken for tests: 0.941354 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 169000 bytes
HTML transferred: 24000 bytes
Requests per second: 1062.30 [#/sec] (mean)
Time per request: 0.941 [ms] (mean)
Time per request: 0.941 [ms] (mean, across all concurrent requests)
Transfer rate: 175.28 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.1 0 10
Processing: 0 0 3.0 0 20
Waiting: 0 0 2.9 0 20
Total: 0 0 3.2 0 20
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 0
95% 10
98% 10
99% 10
100% 20 (longest request)
More information about the Seaside
mailing list