[squeak-dev] Can I create a 75 Gb Image? (about those post-build stats)
gettimothy
gettimothy at zoho.com
Sun Oct 17 18:41:49 UTC 2021
heh..
uptime 0m 5s (runtime 0m 2s, idletime 0m 2s)
memory 635,437,056 bytes
old 96,954,144 bytes (15.3%)
young 528,447,488 bytes (83.2%)
used -237,651,736 bytes (-37.4%)
free 363,654,048 bytes (57.2%)
GCs 2 (2743.5 ms between GCs 1294 ms runtime between GCs)
full 1 totalling 57 ms (2.2% runtime), avg 57 ms
marking 32 ms (56.1%) avg 32 ms,
compacting 25 ms (43.9%) avg 25 ms
scavenges 1 totalling 0 ms (0% runtime), avg 0 ms
tenures 0
Code compactions
0
this was save and quit then restart.
---- On Sun, 17 Oct 2021 14:08:14 -0400 gettimothy via Squeak-dev <squeak-dev at lists.squeakfoundation.org> wrote ----
SmalltalkImage current vmParameterAt:55 -> 0.33333298563957214
I am trying to figure out where that decimal came from. I tried to find the log for base 2,e and 10 and got nothing...so I am going to set it to .75 i guess.
Here is Extras->VMStatistics before the settings:
uptime 3m 48s (runtime 0m 43s, idletime 3m 5s)
memory 128,974,848 bytes
old 120,022,816 bytes (93.1%)
young 7,190,528 bytes (5.6%)
used 97,823,320 bytes (75.8%)
free 27,227,304 bytes (21.1%)
GCs 126 (1811.3 ms between GCs 342.7 ms runtime between GCs)
full 4 totalling 273 ms (0.63% runtime), avg 68.3 ms
marking 129 ms (47.3%) avg 32.3 ms,
compacting 144 ms (52.7%) avg 36 ms
scavenges 122 totalling 57 ms (0.13% runtime), avg 0.5 ms
tenures 92,778 (avg 760 tenures per scavenge)
Code compactions
12 totalling 11 ms (0.025% runtime), avg 0.9 ms
Since last view 106 (1478 ms between GCs, 307 ms runtime between GCs)
uptime 156.8 s, runtime 32.6 s, idletime 124.2 s
full 3 totalling 215 ms (0.7% runtime), avg 71.7 ms
scavenge 103 totalling 47 ms (0.1% runtime), avg 0.5 ms
tenures 13,351 (avg 129 tenures per scavenge)
code compactions 12 totalling 11 ms (0.034% runtime), avg 0.9 ms
here are the settings
SmalltalkImage current vmParameterAt:25 16777216
SmalltalkImage current vmParameterAt:25 put: (2 raisedTo:30).
SmalltalkImage current vmParameterAt:45 0
SmalltalkImage current vmParameterAt:45 put:536870912.
SmalltalkImage current vmParameterAt:55 0.33333298563957214
SmalltalkImage current vmParameterAt:55 put: 0.75
here are the VM Statistics post settings:
uptime 1h 0m 20s (runtime 6m 53s, idletime 53m 27s)
memory 145,752,064 bytes
old 136,800,032 bytes (93.9%)
young 7,190,528 bytes (4.9%)
used 94,899,280 bytes (65.1%)
free 45,887,768 bytes (31.5%)
GCs 1,206 (3001.9 ms between GCs 342.6 ms runtime between GCs)
full 11 totalling 709 ms (0.17% runtime), avg 64.5 ms
marking 354 ms (49.9%) avg 32.2 ms,
compacting 355 ms (50.1%) avg 32.3 ms
scavenges 1,195 totalling 634 ms (0.15% runtime), avg 0.5 ms
tenures 118,269 (avg 98 tenures per scavenge)
Code compactions
35 totalling 33 ms (0.008% runtime), avg 0.9 ms
Since last view 1,080 (3140 ms between GCs, 342 ms runtime between GCs)
uptime 3392.1 s, runtime 369.9 s, idletime 3022.1 s
full 7 totalling 436 ms (0.1% runtime), avg 62.3 ms
scavenge 1,073 totalling 577 ms (0.2% runtime), avg 0.5 ms
tenures 25,491 (avg 23 tenures per scavenge)
code compactions 23 totalling 22 ms (0.006% runtime), avg 1 ms
HOWEVER...your wrote:
I’d be interested to see the vm stats at the end of a build. Those will help you tune and find out how much time is being spent in GC.
Does that mean I must do the whole VMMaker thing?
If so, please point me (again) to the instructions on that and I will get that done this week.
Thanks for your time,
t
---- On Sun, 17 Oct 2021 12:14:41 -0400 Eliot Miranda <mailto:eliot.miranda at gmail.com> wrote ----
Hi Timothy,
On Oct 17, 2021, at 8:41 AM, gettimothy via Squeak-dev <mailto:squeak-dev at lists.squeakfoundation.org> wrote:
Hi Folks,
I am thinking about the result of parsing that 73Gb xml file and storing many of its elements in squeak as live objects.
Assuming an ~1:1 relationship betwen that file and the resultant image, can Squeak handle that?
Can I tell squeak to "SmalltalkImage getLarge:73Gb" ?
I think It would be a fun experiment if this possible.
thx in advance.
In theory there should be no problem other than a rather sedate snapshot and start up time. If your machine has enough ram and disc Spur will save and restore that size of image. You may find that increasing the size of new space/eden to 128/256/512mb, increasing the old space segment size to 1gb, and especially changing the growth to size GC ratio, decreases build time substantially.
About Squeak is your friend. Under vm parameters you’ll find info on sizes and rooting around you’ll find the levers for setting these sizes through vmParameterAt:put:. See parameters 25, 45, & 55. #25 is badly described. It is actually the minimum old space segment size. Setting this to eg 1gb means a lot less segments to traverse than the default 16mb.
I’d be interested to see the vm stats at the end of a build. Those will help you tune and find out how much time is being spent in GC.
_,,,^..^,,,_ (phone)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20211017/96e1d306/attachment-0001.html>
More information about the Squeak-dev
mailing list
|