[VM][BUG] issues with grow/shrink and forward blocks
John M McIntosh
johnmci at smalltalkconsulting.com
Thu Dec 4 05:24:44 UTC 2003
I've spent the afternoon looking at the forwarding block issue. Now if
someone who can remember what
their intent was they should comment. I do recall someone (Ted K?)
reported to me that a largish image reported
odd (slow) behavior when we migrated to VMs that grew and shrunk
memory. Now I know why...
First the important things to remember is that the three important
memory indicators are
freeBlock ------------ endOfMemory -----------------memoryLimit
-------------
My image has for example a million object in it and I invoke
| x |
x _ OrderedCollection new.
1000000 timesRepeat: [x add: Object new].
^x
When we startup on the mac, because of where memoryLimit is first
calculated (out at 512MB of virtual memory) we get 64,284,084 entries
in the forwarding table (between freeBlock and memoryLimit), but not
for long because the shrinkMemory logic promptly drops us back to
531,005 forward table entries between freeBlock start and memoryLimit.
This is the first issue/flaw.
After allocating objects we decrement down to 212,827 free forward
table entries and get a fullGC event because of space issue. The Sweep
Phase then runs but runs out of forward table entries and shifts the
start of the the compaction because it has insufficient forwarding
blocks.
After the sweep this causes a grow to ensure memoryLimit-freeBlock will
contain a number for forward block needed for the newly calculated
totalObjectCount. based on
delta = ((foo->totalObjectCount - sz) + 10000) * 8;
note the *8. Ah but then we invoke initializeMemoryFirstFree and it
uses totalObjectCount as = to number of forward bytes available and
then sets endOfMemory, back say 1,642,476 bytes from memoryLimit. This
then causes
endOfmemory-freeBlock to exceed the shrink threshold (but we just
grew!) and it then shrinks us back to a lower memory allocation right
after allocating all those bytes which results in only for about
500,000 or so forward table entry and the cycle starts again.
Now I believe the issue is the * 8, which is missing in
initializeMemoryFirstFree which then causes this funky grow/shrink
cycle.
When I alter initializeMemoryFirstFree , then I get 1,697,785 forward
table entries after the grow, which then decrements to 1,177,926 before
a fullGC with totalObjectCount of 1,152,791, which I think is the
expected results. Interestingly enough it's possible now to get two
grow operations in a row, one to satisfy the need for more forward
table bytes, then other to deal with growth because of image growth
which was being masked by the original grow/shrink. In my example the
trigger usually was because of having to double in size the ordered
collection object.
Anyway it's nice and complicated and I'm only assuming the original
thought was to ensure the number of forward block entries should match
totalObjectCount and be hidden between endOfMemory and memoryLimit..
--
========================================================================
===
John M. McIntosh <johnmci at smalltalkconsulting.com> 1-800-477-2659
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
========================================================================
===
More information about the Squeak-dev
mailing list
|