[Vm-dev] Memory management in HydraVM

[Igor Stasenko]

2009/2/18 Andreas Raab <andreas.raab at gmx.de>:
>
> Hi Igor -
>
> The main reason for reserving a large chunk of address space is to avoid the
> C allocator to interfere with our need of continuous address space. This
> doesn't that the address space has to be used by a single image.
>
> Since all of the object memory are relocatable, it is easy to see that if
> necessary an object memory could be relocated to a different location. I
> always thought that a Hydra memory managing scheme should work like here:
>
> As a base rule, recursively divide the address space for any new object
> memory. Meaning that with 512MB address space the first image starts at 0,
> the second at 256k the third 128, the fourth at 384 etc. Obviously, if that
> memory is already occupied you move on to the next smaller chunk (i.e., if
> the first image needs more than 256 MB you try 384 next, if that's taken you
> try 448 and so on). Assuming that there is enough base memory to load the
> image (i.e., you aren't trying to load an image into a fragmented space that
> doesn't have sufficient memory at the given location) you will be able to
> load and start that image.
>
> If an image happens to need more space than currently reserved, move the
> adjacent image away. The idea is that you only need to lock and move one
> image (instead of all interpreters) if this is the only one you'll be
> relocating. If there is no space to move the adjacent image, then some
> heuristics needs to be found to figure out what to do. In the early stages I
> would just fail the allocation and give up.
>
> I think the above should work well for most cases. It will deal with one of
> the most important cases: One big master image and many small worker images.
> As the big image grows it will move the smaller images out of the way.
>

aha, you mean that bigger image 'pushing' away smaller ones. But then
if we having, lets say 10 images, where each is 2x bigger than next
one, then if first one demands more address space, we now need to
relocate all 9 images?

Maybe it is better to use different strategy:
allocate the least space for image, then if outage happens - reserve
new address space which size is *2 of previous one and place image
there, and release previously reserved address space back to OS.
A total amount of physical memory required for image relocation could
be minimized, if we copy image in chunks of N pages , and immediately
release N pages in old address space.
The drawback of such approach that it could lead to fragmentation of
address space. But we could limit the minimum amount of address space
for a single interpreter, lets say - 4Mb, so potentially we can afford
about 1024 heaps of size < 4Mb. Which i think is more than enough for
32bit based systems.
In any case, both variants require use of object memory relocation at
some point, its good that VM having this from a very starting, since
loading image into memory sometimes requires relocation if it loaded
into different base address.

As for 64bit systems.. hehe.. i don't think we have to bother with
address space in nearest 10 years.


> Cheers,
>  - Andreas
>

-- 
Best regards,
Igor Stasenko AKA sig.