[Vm-dev] instantiateClass:indexableSize: upper limit?

[Eliot Miranda]

Hi Tom,

On Fri, Jun 23, 2017 at 1:35 AM, Tom Beckmann <tomjonabc at gmail.com> wrote:

>
> Dear VM devs,
>
> I'm trying to allocate a Bitmap of around 16MB size in a plugin via
> `interpreterProxy instantiateClass: interpreterProxy classBitmap
> indexableSize: size`.
> However, I always get NULL back. I already tried wrapping the primitive
> call in `retryWithGC:until:`, with the same effect. Running a normal
> `Bitmap new: 16000000` from a workspace works flawlessly.
>
> I'm on a Ubuntu 16.04 64bit, running a squeak 32bit built from trunk.
>
> Is there anything to watch out for? More precisely, I'm trying to copy the
> contents of an image buffer generated from a pdf via libpoppler at 144dpi
> to a new bitmap in squeak. The primitive and how I invoke it can be seen
> here: https://pastebin.com/9NCdnx2J (line 50 is the call in question)
>
> Thanks in advance for any pointers!
>

I want to endorse what Bert said; allocating in the image is a much better
approach.  But I did want to explain what you're seeing, assuming you're
using Spur (Squeak 5.x, Pharo 6).

The Spur old space heap is segmented.  It grows in increments defined by vm
parameter 25 (Smalltalk vmParameterAt:[put:]), and shrinks whenever free
memory is above parameter 24 and GC empties one or more segments.  It will
not grow memory beyond parameter 67, if that parameter is non-zero:

24 memory threshold above which to shrink object memory (read-write)
25 memory headroom when growing object memory (read-write)
67 the maximum allowed size of old space in bytes, 0 implies no internal
limit (Spur only).

Parmeter 25 defaults to 16Mb (grow in increments of 16Mb) and 24 to 32 Mb
(only shrink while 32Mb or more of empty segments exist).

Currently memory will /only/ grow in these circumstances
- under image control via growMemoryByAtLeast:
- when scavenging tenures sufficient objects to old space that old space
must grow
- when a fullGC fails to ensure there is at least vm parameter 25's worth
of free space
- when an allObjects or allInstances primitive needs more room to answer
its result

The current allocator does /not/ try and grow memory in the allocation
routines (which are accessed in plugins via instantiateClass:indexableSize:
and in the image via new: basicNew: et al), failing instead.  This gives
the image the chance to mediate GC and heap growth (see e.g.
Behavior>>handleFailing[Failing]BasicNew:).

Perhaps instantiateClass:indexableSize: /should/ allow growth but I prefer
it remaining the same.  You would then rewrite your primitive to
- when it fails to allocate memory via instantiateClass:indexableSize: fail
with interpreterProxy primitiveFailFor: PrimErrNoMemory
- have the Smalltalk primitive failure code check the primitive error code
(see senders of primitive:module:error:) and if the failure code is
#'insufficient object memory' grow memory by the necessary amount and retry
the primitive (a la handleFailingBasicNew:)


To the general audience, I think information like the above is key to being
able to understand and exploit the system effectively, but where should it
reside?  Clearly "in my head" is not satisfactory.  It belongs somewhere in
the image, but it needs to be somewhere where people can find it and/or
will look.  Suggestions for an "architectural information" documentation
section gratefully received.  Object class>>whatIsAPrimitive might perhaps
work or perhaps be overloaded.

_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20170623/96df7861/attachment.html>