[squeak-dev] Can I create a 75 Gb Image?

gettimothy gettimothy at zoho.com
Sun Oct 17 17:00:33 UTC 2021


found it:



vmParameterAt: parameterIndex put: newValue

"parameterIndex is a positive integer corresponding to one of the VM's internal

parameter/metric registers.  Store newValue (a positive integer) into that

register and answer with the previous value that was stored there.

Fail if newValue is out of range, if parameterIndex has no corresponding

register, or if the corresponding register is read-only.



As of late 2020 the parameters which can be set are

5	allocations between GCs (read-write; nil in Cog VMs)

6	survivor count tenuring threshold (read-write)

11	tenures of surving objects since startup/last write (read-write)

12	event trace mask; if 1 << eventType is set in the mask then event received by primGetNextEvent: will be printed to stderr.

17	proportion of code zone available for use (Sista VMs only)

23	bytes of extra memory to reserve for VM buffers, plugins, etc.

24	memory threshold above whichto shrink object memory (read-write)

25	memory headroom when growing object memory (read-write)

26	interruptChecksEveryNms - force an ioProcessEvents every N milliseconds (read-write)

34	bytes allocated in total since start-up or reset (read-write)

43	desired number of stack pages (stored in image file header, max 65535; Cog VMs only, otherwise nil)

45	desired size of eden, in bytes (stored in image file header; Cog VMs only, otherwise nil)

47	desired size of machine code zone, in bytes (applies at startup only, stored in image file header; Cog JIT VM only)

48	various properties of the Cog VM as an integer encoding an array of bit flags.

Bit 0: tells the VM that the image's Process class has threadId as its 5th inst var (after nextLink, suspendedContext, priority & myList)

Bit 1: on Cog JIT VMs asks the VM to set the flag bit in interpreted methods

Bit 2: if set, preempting a process puts it to the head of its run queue, not the back,

i.e. preempting a process by a higher priority one will not cause the preempted process to yield

to others at the same priority.

Bit 3: in a muilt-threaded VM, if set, the Window system will only be accessed from the first VM thread

Bit 4: in a Spur vm, if set, causes weaklings and ephemerons to be queued individually for finalization

49	the size of the external semaphore table (read-write; Cog VMs only)

55	ratio of growth and image size at or above which a GC will be performed post scavenge (Spur only, otherwise nil)

67	the maximum allowed size of old space in bytes, 0 implies no internal limit (Spur only).

68	the average number of live stack pages when scanned by GC (at scavenge/gc/become et al)

69	the maximum number of live stack pages when scanned by GC (at scavenge/gc/become et al)

74 maximum pause time due to segment allocation

75 whether arithmetic primitives will do mixed type arithmetic; if false they fail for different receiver and argument types"





<primitive: 254>

self primitiveFailed




<puts on mad scientist hat>





thx.



---- On Sun, 17 Oct 2021 12:14:41 -0400 Eliot Miranda <eliot.miranda at gmail.com> wrote ----


Hi Timothy,
On Oct 17, 2021, at 8:41 AM, gettimothy via Squeak-dev <mailto:squeak-dev at lists.squeakfoundation.org> wrote:



Hi Folks,





I am thinking about the result of parsing that 73Gb xml file and storing many of its elements in squeak as live objects.



Assuming an ~1:1 relationship betwen that file and the resultant image, can Squeak handle that?



Can I tell squeak to "SmalltalkImage getLarge:73Gb"  ? 





I think It would be a fun experiment if this possible.



thx in advance.






In theory there should be no problem other than a rather sedate snapshot and start up time.  If your machine has enough ram and disc Spur will save and restore that size of image.  You may find that increasing the size of new space/eden to 128/256/512mb, increasing the old space segment size to 1gb, and especially changing the growth to size GC ratio, decreases build time substantially.

About Squeak is your friend.  Under vm parameters you’ll find info on sizes and rooting around you’ll find the levers for setting these sizes through vmParameterAt:put:.  See parameters 25, 45, & 55. #25 is badly described.  It is actually the minimum old space segment size.  Setting this to eg 1gb means a lot less segments to traverse than the default 16mb.



I’d be interested to see the vm stats at the end of a build. Those will help you tune and find out how much time is being spent in GC.


_,,,^..^,,,_ (phone)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20211017/9769b20b/attachment.html>


More information about the Squeak-dev mailing list