[ENH] WeakIdentityDictionary hashing ( [cd][er][sm][et][approved] Good for the future )

David T. Lewis lewis at mail.msen.com
Fri Jul 4 14:32:02 UTC 2003


On Wed, Jul 02, 2003 at 05:35:48PM -0800, ned at bike-nomad.com wrote:
> 
> Though the current image does not use large WeakIdentityKeyDictionaries,
> heavy use of the new events system or other code needing large
> registries could benefit from this.
> 
> The downside for small dictionaries is a single comparison of the size
> with 4096 per access. This is more than made up for by speed increases
> on big dictionaries.
> 
> I think this is worth it.

I found out a while back (while working on CommandShell testing in
a Squeak 3.2 image) that performance of the registry for StandardFileStream
degrades horribly when the registry gets large. The time was being used
in WeakKeyDictionary>>scanForNil:

I don't know if Andreas' change set directly effects this, but I can
say that anything the improves performance #scanForNil: for a large
collection is probably a really good thing.

Dave

FWIW, here is an example time profile from one of my old 3.2 test images.
I have a workaround for the problem so it is not an issue for me any more,
the but profile is interesting. In this case, 80% of the time was spent
in #scanForNil: while removing entries from the registry.

---------

 - 469 tallies, 8147 msec.

**Tree**
94.2% {7674ms} CommandShell>>pipeline:
  |64.0% {5214ms} PipeableOSProcess>>value
  |  |17.7% {1442ms} ExternalUnixOSProcess(ExternalOSProcess)>>closeInitialStdIn
  |  |  |17.7% {1442ms} AttachableFileStream(StandardFileStream)>>close
  |  |  |  17.7% {1442ms} AttachableFileStream(StandardFileStream)>>unregister
  |  |  |    17.7% {1442ms} AttachableFileStream class(StandardFileStream class)>>unregister:
  |  |  |      17.7% {1442ms} WeakRegistry>>remove:ifAbsent:
  |  |  |        17.7% {1442ms} WeakKeyDictionary(Dictionary)>>removeKey:ifAbsent:
  |  |  |          17.7% {1442ms} WeakKeyDictionary>>fixCollisionsFrom:
  |  |  |            17.7% {1442ms} WeakKeyDictionary>>rehash
  |  |  |              17.5% {1426ms} WeakKeyDictionary>>scanForNil:
  |  |17.7% {1442ms} ExternalUnixOSProcess(ExternalOSProcess)>>closeInitialStdOut
  |  |  |17.7% {1442ms} AttachableFileStream(StandardFileStream)>>close
  |  |  |  17.7% {1442ms} AttachableFileStream(StandardFileStream)>>unregister
  |  |  |    17.7% {1442ms} AttachableFileStream class(StandardFileStream class)>>unregister:
  |  |  |      17.7% {1442ms} WeakRegistry>>remove:ifAbsent:
  |  |  |        17.7% {1442ms} WeakKeyDictionary(Dictionary)>>removeKey:ifAbsent:
  |  |  |          17.7% {1442ms} WeakKeyDictionary>>fixCollisionsFrom:
  |  |  |            17.7% {1442ms} WeakKeyDictionary>>rehash
  |  |  |              17.1% {1393ms} WeakKeyDictionary>>scanForNil:
  |  |17.3% {1409ms} ExternalUnixOSProcess(ExternalOSProcess)>>closeInitialStdErr
  |  |  |17.3% {1409ms} AttachableFileStream(StandardFileStream)>>close
  |  |  |  17.3% {1409ms} AttachableFileStream(StandardFileStream)>>unregister
  |  |  |    17.3% {1409ms} AttachableFileStream class(StandardFileStream class)>>unregister:
  |  |  |      17.3% {1409ms} WeakRegistry>>remove:ifAbsent:
  |  |  |        17.3% {1409ms} WeakKeyDictionary(Dictionary)>>removeKey:ifAbsent:
  |  |  |          17.3% {1409ms} WeakKeyDictionary>>fixCollisionsFrom:
  |  |  |            17.3% {1409ms} WeakKeyDictionary>>rehash
  |  |  |              16.2% {1320ms} WeakKeyDictionary>>scanForNil:
  |  |11.3% {921ms} ExternalUnixOSProcess>>value
  |  |  11.3% {921ms} ExternalUnixOSProcess>>forkChild
  |  |    11.3% {921ms} UnixProcess>>processProxy:forkAndExe...nvironment:descriptors:
  |  |      11.1% {904ms} Semaphore>>critical:
  |30.3% {2469ms} CommandShell>>proxyPipelineFrom:
  |  11.9% {969ms} PipeableOSProcess>>prepareOutputForPipe
  |    |11.9% {969ms} PipeableOSProcess(PipeJunction)>>prepareOutputForPipe
  |    |  11.9% {969ms} PipeableOSProcess>>enableErrorStreamTriggers
  |    |    11.9% {969ms} UnixOSPipe>>setBufferedReader
  |    |      11.9% {969ms} AttachableFileStream(StandardFileStream)>>unregister
  |    |        11.9% {969ms} AttachableFileStream class(StandardFileStream class)>>unregister:
  |    |          11.9% {969ms} WeakRegistry>>remove:ifAbsent:
  |    |            11.9% {969ms} WeakKeyDictionary(Dictionary)>>removeKey:ifAbsent:
  |    |              11.9% {969ms} WeakKeyDictionary>>fixCollisionsFrom:
  |    |                11.9% {969ms} WeakKeyDictionary>>rehash
  |    |                  11.9% {969ms} WeakKeyDictionary>>scanForNil:
  |  11.7% {953ms} PipeableOSProcess>>prepareOutputForInternalReader
  |    |11.7% {953ms} PipeableOSProcess(PipeJunction)>>prepareOutputForInternalReader
  |    |  11.7% {953ms} PipeableOSProcess>>enableEventTriggers
  |    |    6.0% {489ms} PipeableOSProcess>>enableOutputStreamTriggers
  |    |      |6.0% {489ms} UnixOSPipe>>setBufferedReader
  |    |      |  6.0% {489ms} AttachableFileStream(StandardFileStream)>>unregister
  |    |      |    6.0% {489ms} AttachableFileStream class(StandardFileStream class)>>unregister:
  |    |      |      6.0% {489ms} WeakRegistry>>remove:ifAbsent:
  |    |      |        6.0% {489ms} WeakKeyDictionary(Dictionary)>>removeKey:ifAbsent:
  |    |      |          6.0% {489ms} WeakKeyDictionary>>fixCollisionsFrom:
  |    |      |            6.0% {489ms} WeakKeyDictionary>>rehash
  |    |      |              5.8% {473ms} WeakKeyDictionary>>scanForNil:
  |    |    5.8% {473ms} PipeableOSProcess>>enableErrorStreamTriggers
  |    |      5.8% {473ms} UnixOSPipe>>setBufferedReader
  |    |        5.8% {473ms} AttachableFileStream(StandardFileStream)>>unregister
  |    |          5.8% {473ms} AttachableFileStream class(StandardFileStream class)>>unregister:
  |    |            5.8% {473ms} WeakRegistry>>remove:ifAbsent:
  |    |              5.8% {473ms} WeakKeyDictionary(Dictionary)>>removeKey:ifAbsent:
  |    |                5.8% {473ms} WeakKeyDictionary>>fixCollisionsFrom:
  |    |                  5.8% {473ms} WeakKeyDictionary>>rehash
  |    |                    5.5% {448ms} WeakKeyDictionary>>scanForNil:
  |  6.6% {538ms} CommandShell>>redirectedPipeableProxyFor:predecessorProxy:
  |    6.6% {538ms} CommandShell>>pipeableProxyFor:input:...rror:predecessorProxy:
  |      6.6% {538ms} CommandShell>>processProxyFor:input:o...rror:predecessorProxy:
  |        6.2% {505ms} AttachableFileStream(StandardFileStream)>>close
  |          6.2% {505ms} AttachableFileStream(StandardFileStream)>>unregister
  |            6.2% {505ms} AttachableFileStream class(StandardFileStream class)>>unregister:
  |              6.2% {505ms} WeakRegistry>>remove:ifAbsent:
  |                6.2% {505ms} WeakKeyDictionary(Dictionary)>>removeKey:ifAbsent:
  |                  6.2% {505ms} WeakKeyDictionary>>fixCollisionsFrom:
  |                    6.2% {505ms} WeakKeyDictionary>>rehash
  |                      6.0% {489ms} WeakKeyDictionary>>scanForNil:
5.8% {473ms} PipeableOSProcess>>output
  5.8% {473ms} PipeableOSProcess>>outputWithErrorNotifier:
    4.9% {399ms} PipeableOSProcess>>upToEnd
      4.9% {399ms} Semaphore>>critical:

**Leaves**
80.0% {6518ms} WeakKeyDictionary>>scanForNil:
16.0% {1304ms} Semaphore>>critical:

**Memory**
	old			+0 bytes
	young		-49,824 bytes
	used		-49,824 bytes
	free		+49,824 bytes

**GCs**
	full			0 totalling 0ms (0.0% uptime)
	incr		9 totalling 41ms (1.0% uptime), avg 5.0ms
	tenures		0
	root table	0 overflows




More information about the Squeak-dev mailing list