On Wed, Jan 12, 2005 at 11:09:17PM -0800, Dan Ingalls wrote:
> Ian and I were talking today about simple storage management schemes and realized that Squeak's current garbage collector is "so wonderful" that it has shielded us completely from a useful piece of information.
> 
> We were specifically curious how a non-moving garbage collector would perform.  For such a beast, when an object is freed it is maintained in a structure from which it can later be reallocated.  Most such approaches use some form of "buddy" system to coalesce adjacent free blocks but, inevitably there is some fragmentation of memory.
> 
> So here's the instrumentation challenge:  Write a log of all the allocation and release operations from running Squeak through, say, the macro benchmarks.
> 
> There's a pretty simple place to find all the allocations in the VM.  Then at each GC, each unmarked object that the compactor steps over should cause an effective release operation to be logged.
> 
> Such a log could then be fed to any number of block management algorithms to get a sense of what kind of fragmentation to expect.  This, in turn, would tell us what kind of trade-off exists between the memory footprint used and the time between catastrophic reorganization (ie when you have to do a full compaction).
> 
> Partial credit:  If you have real experience with a Squeak-like system that worked this way, give some report on the fragmentation behavior experienced.
> 
> References to papers with practical results in the area would also be of interest.
> 
> Thanks
> 
> 	- Dan

For my entry in the hotly contested better-late-than-never category, I am
attaching AllocationLoggingPlugin-dtl.cs, a memory allocation logging goodie.
Along with the change set, I've attached the modified sqVirtualMachine.c
and sqVirtualMachine.h files and corresponding diff files (I'm not sure
which is most convenient). These required a new entry to permit the allocation
logging plugin to register its logging function for the interpreter to use.

>From the preamble:

Log memory allocation activity to a file. Caution - may create very large log
files.

  AllocationLogger new log: 'memory.log' while: [Array new: 2000000]
  AllocationLogger new log: 'memory.log' while: [Smalltalk garbageCollect]
  AllocationLogger new log: 'memory.log' while: [MacroBenchmarks run]

The AllocationLoggingPlugin arranges for the interpreter to log interesting
memory allocation and deallocation events to an external log file. The plugin
registers a logging function (#log:with:with) that is called by the object
memory to record events.

Primitives are provided to open and close a log file, and to control what
events are to be included in the log. Records in the log file have comma
separated fields with time stamp (ioMSecs), a memory function code such
as 'Alloc' or 'Free', and two integer values that pertain to the memory
function. See #initializeMemoryLoggingConstants for the memory functions
that can be logged.

This plugin requires an update to the sqVirtualMachine.c and
sqVirtualMachine.h to define the logging function.

In sqVirtualMachine.c:

#if VM_PROXY_MINOR > 6
        VM->memoryLoggingFunction= NULL; /* set by a plugin */
#endif

In sqVirtualMachine.h:

#define VM_PROXY_MINOR 7

#if VM_PROXY_MINOR > 6
        /* function to be provided by a plugin */
        int (*memoryLoggingFunction)(int code, int param1, int param2);
#endif

Note: #allocateChunk: also contains a change for a low space interrupt fix,
commented out for this AllocationLoggingPlugin distribution. The two change
sets will need to be merged.

Dave

-------------- next part --------------
A non-text attachment was scrubbed...
Name: AllocationLoggingPlugin-dtl.cs.gz
Type: application/x-gunzip
Size: 10075 bytes
Desc: not available
Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20050502/98038e96/AllocationLoggingPlugin-dtl.cs.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sqVirtualMachine.c.diff.gz
Type: application/x-gunzip
Size: 146 bytes
Desc: not available
Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20050502/98038e96/sqVirtualMachine.c.diff.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sqVirtualMachine.h.diff.gz
Type: application/x-gunzip
Size: 201 bytes
Desc: not available
Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20050502/98038e96/sqVirtualMachine.h.diff.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sqVirtualMachine.c.gz
Type: application/x-gunzip
Size: 2527 bytes
Desc: not available
Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20050502/98038e96/sqVirtualMachine.c.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sqVirtualMachine.h.gz
Type: application/x-gunzip
Size: 1847 bytes
Desc: not available
Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20050502/98038e96/sqVirtualMachine.h.bin