[Vm-dev] Re: [squeak-dev] What is the upper memory limits of Cog?

[Chris Muller]

On Thu, Sep 25, 2014 at 4:17 PM, Eliot Miranda <eliot.miranda at gmail.com> wrote:
>
> Hi Chris,
>
>     it depends...
>
> First of all, 32-bit linux only provides 3Gb of address space to user processes, the upper quarter being used to map the kernel into the user's address space.  Look at the stack pointer and you'll see it starts at 0xbfffxxxx, not 0xffffxxxx.  Apparently there are ways around this but I can't find good references for this.  e.g. this is an unhelpful page, talking about total system address space, not per-process address space: http://en.wikipedia.org/wiki/3_GB_barrier.
>
> Second of all, the VM shares the address space with other components in the program, especially X11 which uses a lot of address space.  Running headless should allow you to gro a little bit more.
>
> In my recent experience on linux one can indeed start Cog V3 with about 1.9Gb.  Spur can grow to about 2.9Gb.  The V3 memory manager requires a contiguous heap, which forces preallocation.  This is a serious problem, but one that Spur solves.  Spur has a segmented memory model, and so can grow the heap dynamically.  But the fundamental problem is that linux only provides 3Gb per process.  So until one has a 64-bit Spur heap size will be limited to 2.9Gb or there abouts on linux.
>
> Is this clear enough?

Totally.  Thanks.

> For more, see my message announcing the availability of the 3018 VMs.  From that message I wrote:
>
> Here's my test script:
> | them |
> them := OrderedCollection new.
> [[them addLast: (ByteArray new: 16000000).
>  Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush] repeat]
> on: OutOfMemory
> do: [:ex| 2 to: them size by: 2 do: [:i| them at: i put: nil. Smalltalk garbageCollect]].
> Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush.
> them := nil.
> Smalltalk garbageCollect.
> Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush

I have had to put memory-checking into my code and terminate "stale"
resources when it hits the ceiling.  Part of my memory-checking logic
relies on (Smalltalk vmParameterAt: 3) to estimate available memory,
but since it is returning -868260916 (that's right, --negative--) for
that value, my assumptions that it would always be positive were wrong
and that explains why the memory-checking is not working correctly.

Is something possibly wrong that this is negative?  Am I supposed to
take the absolute value of it?