AW: Dynamic system memory use

Andreas Raab Andreas.Raab at gmx.de
Fri Feb 1 20:29:24 UTC 2002


Marcel,

> Also, I started implementing the grow/shrink primitives, but then 
> couldn't figure out how such an implementation differs from simply 
> starting Squeak with -memory 512, in the presence of a lazy 
> VM subsystem.

It differs in various ways. First of all, let me repeat what's actually
the expected behavior of any implementation for dynamic memory
management functions: 

On startup we _reserve_ an (arbitrarily high) amount of address space.
The only reason we do this is to prevent the stupid C allocator from
giving away memory with addresses in the specified region so that Squeak
is guarantueed to get a linear chunk if requested. At this point, we
don't take a single byte of the memory, all we do is _reserving address
space_. When we call one of the shrink/grow functions, then we actually
request (continuous) portions of memory out of the reserved address
space. Now we need it and we know we need it. Growing may fail due to
insufficient resources and that's fine; we can recover from such a
failure quite nicely (e.g., you'll just get a low space warning).

Compare that to your lazy VM subsystem: For one thing, your VM subsystem
may require backing store for the entire reserved address space which is
a significant waste of disk space (the current Windows VM reserves about
1.5GB of memory and I tend to run multiple images at the same time).
Secondly, if it does _not_ provide backing store, then your VM subsystem
may randomly fail at some unspecified point of actually accessing the
memory it promised to you. Meaning that instead of a low space warning
you'll get a core dump (if at all). Thirdly, since the VM subsystem can
hardly monitor whether we still write to pages of memory (which, after a
shrink we do _not_) it is unable to release that portion of memory to
the OS or other apps. Fourthly, since Squeak has no knowledge about how
much memory is actually available from the system without going to
lengthy swapping, there is no way of optimizing large data set
manipulations based on the amount of physical memory (which can be a big
advantage due to the unusually low localization of memory access
patterns in OOP systems). So there's a _huge_ difference between a lazy
VM subsystem and the behavior described above.

What I would recommend is looking at your common C allocator and find a
way of reserving address space (is there a function to create a new heap
in Linux?) and allocate/free continous portions from it (used to be
called sbrk()).

Cheers,
  - Andreas





More information about the Squeak-dev mailing list