Dynamic system memory use
John M McIntosh
johnmci at smalltalkconsulting.com
Fri Feb 1 07:59:49 UTC 2002
>Tim Rowledge <tim at sumeru.stanford.edu> wrote:
>> Squeak can increase the memory it uses if the OS supports a suitable
>> method of extending the original allocation - or potentially making a
>> new larger allocation and copying the entire object memory across and
>> swizzling all the pointers. See the macros
>> #define sqGrowMemoryBy(oldLimit, delta) oldLimit
>> #define sqShrinkMemoryBy(oldLimit, delta) oldLimit
>> #define sqMemoryExtraBytesLeft(includingSwap) 0
>> in sq.h and their uses.
>
>Unfortunately, these functions are hard to implement. They expect the
>heap to be reallocated *in place*, and more importantly, they expect the
>heap to stay as one contiguous segment. realloc() could probably be
>made to work, but it's still going to be difficult on the underlying OS,
>simply because realloc-ing hundreds of megs will make the 4 gigs of
>32-bit addressable space start running out!
Actually because of real pointers and how the VM works, it's only
2GB, the first 2.
>
>I don't know what is being done on the Windows and MacOS ports to
>support these. On Unix, all I could think of is to allocate a huge
>amount of space using mmap, and then not touch it until it's needed.
>This works for allocation. For deallocation, munmap can be used,
>followed by an immediate mmap. Unfortunately, there is no real
>guarantee that the second mmap is going to work, which would mean the
>system suddenly has a very small amount of free heap space!
>
For OS-X I mmap as anonymous 512MB. I then move the end of memory
location within those 512MB. If the memory isn't used it never gets
allocated. If the memory gets used, and is paged out to disk, *and*
we shrink the image size, well I just ignore it and have the virtual
memory manager leave those inactive pages sit on the disk. If we
grow, then the cost is mapping those pages in to be overwritten, but
the cost is no different than taking a page fault to get more memory
in a malloc situation. (but see my note about madvise)
So mmap and forget about an ummap and mmap.
Lets see..
PhysMem: 60.2M wired, 78.6M active, 412M inactive, 551M used, 217M free
VM: 3.03G + 55.3M 7847(0) pageins, 1(0) pageouts
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
422 Squeak VM 0.8% 0:35.59 2 71 100 23.3M 8.53M 26.5M 571M
but I only have one 80,000,000 byte swap file in /var/vm
after allocating 400K bytearray, with another squeak vm, so it's not
quite before/after
PhysMem: 64.0M wired, 55.7M active, 639M inactive, 759M used, 9.39M free
VM: 3.21G + 50.5M 16138(13) pageins, 5904(0) pageouts
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
1235 Squeak VM 1.5% 0:14.43 3 77 119 421M 8.02M 424M 573M
Now I have two 80,000,000 byte swap file in /var/vm
Lets Free and GC
memory 441,367,964 bytes
old 17,540,780 bytes (4.0%)
young 419,621,884 bytes (95.1%)
used 437,162,664 bytes (99.0%)
free 4,205,300 bytes (1.0%)
GCs 395 (482ms between GCs)
full 2 totalling 1,432ms (1.0% uptime), avg 716.0ms
incr 393 totalling 685ms (0.0% uptime), avg 2.0ms
tenures 0
goes to
memory 21,753,888 bytes
old 17,559,608 bytes (80.7%)
young 81,152 bytes (0.4%)
used 17,640,760 bytes (81.1%)
free 4,113,128 bytes (18.9%)
GCs 505 (424ms between GCs)
full 3 totalling 2,123ms (1.0% uptime), avg 708.0ms
incr 502 totalling 900ms (0.0% uptime), avg 2.0ms
tenures 0
MemRegions: num = 3635, resident = 498M + 6.66M private, 57.3M shared
PhysMem: 64.0M wired, 58.2M active, 634M inactive, 756M used, 12.1M free
VM: 3.21G + 50.5M 16150(1) pageins, 6433(0) pageouts
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
1235 Squeak VM 9.3% 0:36.43 2 75 117 421M 8.00M 424M 573M
Technically I could do an madvise with MADV_FREE when we shrink to
state the excess pages
aren't needed and can be freed but maybe we should see what the
behavior is first. Mmm
actually doing the MADV_FREE might be a good idea, we'll see for
3.2.3, I suspect it will make the VM subsystem more happy, actually
some reading shows it will make it *very* happy.
Note somehow I miss my Linux box, so I could go do a man madvise and
see if that is supported.
Well I can hunt on the internet but it's not quite the same (which
version is that for?)
I *think* linux is MADV_DONTNEED, but I'm not sure exactly what it
does (conflicting stories).
--
--
===========================================================================
John M. McIntosh <johnmci at smalltalkconsulting.com> 1-800-477-2659
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
===========================================================================
More information about the Squeak-dev
mailing list
|