Squeak FTP much slower than Python

goran.hultgren at bluefish.se goran.hultgren at bluefish.se
Tue May 14 08:56:20 UTC 2002


Ted Kaehler <Ted at SqueakLand.org> wrote:
> Folks,
> 	Someone asked why Squeak is so slow to bring in a large file 
> via FTP.  Here are two easy things that might make it better.
> 
> 	In FTPSocket>>getDataTo: dataStream whileWatching: 
> otherSocket and its five similar methods, (buf _ String new: 4000.) 
> appears.  One could raise that buffer size to something much bigger.
> 	But, there is another bottleneck also.  A Stream is being 
> created, and it grows periodically.  Here is the code that grows a 
> stream:
> 
> WriteStream
> pastEndPut: anObject
> 	collection _ collection ,
> 		(collection class new: ((collection size max: 20) min: 20000)).
> 	writeLimit _ collection size.
> 	collection at: (position _ position + 1) put: anObject
> 
> Note that the size is doubling each time it hits the end, but adds a 
> maximum of 20k.  This was fine in the old days, but it is not good 
> for reading a 4 meg file.   I suggest removing the cap altogether, or 
> cap it at 1 meg added each time.  Each growth causes everything 
> before to be copied again.  4meg / 20k = 200 times that the data is 
> recopied during the transfer.

Sidenote:

Using #, is not so good since it will do an unnecessary copy of all
those nil values in the new empty collection that is "tacked on".
This same "bug" is in Comanche's BufferStream (but BufferStream doesn't
seem to suffer because it rarely needs to grow in Comanche).

It is probably better to do something like:

newCollection _ collection class new: collection size + ((collection
size max: 20) min: 20000).
newCollection
	replaceFrom: 1
	to: collection size
	with: collection
	startingAt: 1.
collection _ newCollection

Well, after testing it out by writing a new #pastEndPut: it didn't
matter that much - obviously other stuff hides this little effect, but
this little test shows the difference - print them both and compare:

| collection newCollection result |
Smalltalk garbageCollect.
collection _ String new: 10000.
Time millisecondsToRun: [
1000 timesRepeat: [
newCollection _ collection class new: collection size + ((collection
size max: 20) min: 20000).
newCollection
		replaceFrom: 1
		to: collection size
		with: collection
		startingAt: 1.
result _ newCollection]]

Alt-p -> 100 95 91

| collection result |
Smalltalk garbageCollect.
collection _ String new: 10000.
Time millisecondsToRun: [
1000 timesRepeat: [
result _ collection , (collection class new: collection size +
((collection size max: 20) min: 20000))]]

Alt-p ->  245 243 254


regards, Göran



More information about the Squeak-dev mailing list