Dynamic system memory use
Marcel Weiher
marcel at metaobject.com
Fri Feb 1 19:01:23 UTC 2002
On Friday, February 1, 2002, at 10:22 , John M McIntosh wrote:
>> Any particular reason for using mmap() and not just simply malloc()?
>> It will have the same semantics of just allocating address-space.
>
> I started with mmap because of the linux work. Also I'm not sure if
> malloc would reserve all 512mb right away for me.
It would. It has to, since it guarantees to return a chunk of
sequential memory. With such a large allocation, you will get
zero-filled VM-pages.
>>
>> One reason for mmap() would be that you can use it to create a private
>> swapfile, which can be significantly better than going through the
>> system swapfiles in some circumstances.
>>
>> Also, I started implementing the grow/shrink primitives, but then
>> couldn't figure out how such an implementation differs from simply
>> starting Squeak with -memory 512, in the presence of a lazy VM
>> subsystem.
>
> Good question, when does the grow happen? Versus if you have 500MB of
> free space.
I am not quite sure what you mean here. There is no real growth, as in
realloc() or the like. As soon as you touch a page (read or write, it
seems, although it could really delay until a write), the kernel will
provide real memory for it, but no sooner.
>>
>>> Technically I could do an madvise with MADV_FREE when we shrink to
>>> state the excess pages
>>> aren't needed and can be freed but maybe we should see what the
>>> behavior is first. Mmm
>>> actually doing the MADV_FREE might be a good idea, we'll see for
>>> 3.2.3, I suspect it will make the VM subsystem more happy, actually
>>> some reading shows it will make it *very* happy.
>>
>> You have to be careful with these. Some are not supported, some don't
>> really work and others give you kernel panics. Apple has some
>> bug-reports from me...
>
> I'd be interested to know what they are, and if you can issue MADV_FREE
> without fear, since it seems the proper way to do things.
The answer is: I am not sure. What I found is that lockups were
triggered under some circumstances that depended on external/variable
factors such as total VM subsystem load etc. While I can describe the
particular circumstances that triggered it for me, I am not at all sure
wether there aren't some other circumstances that would trigger similar
or other unintended behaviors. I simply wouldn't mess with this stuff
right now.
Anyway, the real proper way to do this on Mach would be a user-level
pager. Alas, the user-level pager facility (EMMI, External Memory
Manager Interface) is 'resting' in current Mac OS X. Apparently, some
of the low-level mechanisms it relies on are being revamped, and I was
assured that it would be coming back by one of the kernel engineers.
In the meantime, I think just setting a large memory size and letting
the VM subsystem handle it is probably fully adequate.
Marcel
More information about the Squeak-dev
mailing list
|