<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 10 Jan 2016, at 20:22, <a href="mailto:vm-dev-request@lists.squeakfoundation.org" class="">vm-dev-request@lists.squeakfoundation.org</a> wrote:</div><br class="Apple-interchange-newline"><div class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Hi Max,</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""> pre-Spur to avoid GC one has to a) grow memory by enough to do all the</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">processing you're going to do and b) change the shrinkage parameter so the</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Vm won't shrink the heap back down before the processing is complete. To</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">do b) I suggest you modify setGCParameters. vm parameters 24 sets the</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">shrinkage threshold; see vmParameterAt:put:: "24 memory threshold above</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">whichto shrink object memory (read-write)". growMemory. Hmmm, I had</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">thoguht that there's a growMemoryBy: primitive in v3, but it appears there</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">isn't. So simply allocate a ByteArray of the desired size and then GC to</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">get rid of it. That should leave that much free space and then your load</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">should proceed without needing to GC.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Anyway, it's worth a try.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></blockquote><div><br class=""></div><div>Thanks Eliot.</div><div><br class=""></div><div>Setting the memory threshold helped. I’m still seeing one full GC which I’m trying to avoid. I’ve experimented with #setGCBiasToGrow: and #setGCBiasToGrowGCLimit: but I don’t fully understand what they do. </div><div>#setGCBiasToGrow: seems to turn memory growth on and off. But if this is turned off, how can the VM then allocate more memory?</div><div>#setGCBiasToGrowGCLimit: seems to control if the growth should trigger a full GC, which seems pretty much like what I need.</div><div><br class=""></div><div>Unfortunately, while setting these options seems to have an influence, I can’t quite see the pattern, and that one full GC is still there. Maybe you could explain how these options work exactly?</div><div><br class=""></div><div>One other question: the MessageTally output seems to be missing about 50% of the running time. Summing up over full GC + incremental GC + time spent in the tree, leaves about 500ms unaccounted for. Do you have any idea where that half second goes missing?</div><div><br class=""></div><div><br class=""></div><div>Here’s the code I use experimentally:</div><div><br class=""></div><div><div><div>MessageTally spyOn: [</div></div></div></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div><div><div><div>| size shrinkThreshold |</div></div></div></div><div><div><div><div>size := (self settings segmentsDirectory fileNamed: 'snapshot.bin') size.</div></div></div></div><div><div><div><div>shrinkThreshold := (Smalltalk vmParameterAt: 24).</div></div></div></div><div><div><div><div>Smalltalk vmParameterAt: 24 put: shrinkThreshold + (size*2). "(8MB + twice file size)"</div></div></div></div><div><div><div><div>Smalltalk setGCBiasToGrowGCLimit: shrinkThreshold + (size*2).</div></div></div></div><div><div><div><div>Smalltalk setGCBiasToGrow: 1. “enable growth??"</div></div></div></div><div><div><div><div>ByteArray new: size*2.</div></div></div></div><div><div><div><div><br class=""></div><div>"incremental GC should take care of collecting the ByteArray, so I’m not doing anything</div></div></div></div></blockquote><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div><div><div><div>manually here"</div></div></div></div></blockquote><div><div><div><div><br class=""></div></div></div></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div><div><div><load snapshot> ].</div></div></div></blockquote><div><div><br class=""></div><div><br class=""></div><div><br class=""></div><div>Cheers,</div><div>Max</div><div><br class=""></div><div><br class=""></div><div><br class=""></div><div>Output from current MessageTally:</div><div><br class=""></div><div><div><div> - 1123 tallies, 1125 msec.</div><div><br class=""></div><div>**Tree**</div><div>--------------------------------</div><div>Process: (40s) 123994112: nil</div><div>--------------------------------</div><div>12.7% {143ms} CBImageSegment class(NSImageSegment class)>>basicSnapshot:from:do:</div><div> 12.6% {141ms} CBImageSegment class(NSImageSegment class)>>installSegmentFrom:andDo:</div><div> 12.6% {141ms} CBImageSegment class(NSImageSegment class)>>readSegmentFrom:</div><div> 12.6% {141ms} NSSegmentStream>>readObject</div><div> 12.6% {141ms} SmartRefStream>>nextAndClose</div><div> 12.6% {141ms} SmartRefStream>>next</div><div> 12.3% {138ms} SmartRefStream(ReferenceStream)>>next</div><div> 12.3% {138ms} SmartRefStream(DataStream)>>next</div><div> 10.6% {119ms} CBImageSegment(ImageSegment)>>comeFullyUpOnReload:</div><div> |10.6% {119ms} CBImageSegment(NSImageSegment)>>restoreEndiannessAndRehash</div><div> | 5.5% {62ms} Dictionary>>rehash</div><div> | |2.8% {31ms} Dictionary>>associationsDo:</div><div> | | |2.2% {25ms} Array(SequenceableCollection)>>do:</div><div> | |1.7% {19ms} Dictionary>>noCheckAdd:</div><div> | | 1.7% {19ms} Dictionary(HashedCollection)>>findElementOrNil:</div><div> | | 1.2% {13ms} Dictionary>>scanFor:</div><div> | 4.5% {51ms} primitives</div><div> 1.2% {13ms} SmartRefStream(DataStream)>>readArray</div><div> 1.2% {13ms} SmartRefStream>>next</div><div> 1.2% {13ms} SmartRefStream(ReferenceStream)>>next</div><div> 1.2% {13ms} SmartRefStream(DataStream)>>next</div><div> </div><div>**Leaves**</div><div><br class=""></div><div>**Memory**</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>old<span class="Apple-tab-span" style="white-space:pre"> </span>+94,031,228 bytes</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>young<span class="Apple-tab-span" style="white-space:pre"> </span>-9,207,660 bytes</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>used<span class="Apple-tab-span" style="white-space:pre"> </span>+84,823,568 bytes</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>free<span class="Apple-tab-span" style="white-space:pre"> </span>+90,024,824 bytes</div><div><br class=""></div><div>**GCs**</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>full<span class="Apple-tab-span" style="white-space:pre"> </span>1 totalling 85ms (8.0% uptime), avg 85.0ms</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>incr<span class="Apple-tab-span" style="white-space:pre"> </span>15 totalling 271ms (24.0% uptime), avg 18.0ms</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tenures<span class="Apple-tab-span" style="white-space:pre"> </span>10 (avg 1 GCs/tenure)</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>root table<span class="Apple-tab-span" style="white-space:pre"> </span>0 overflows</div><div class=""><br class=""></div></div></div><br class=""><blockquote type="cite" class=""><div class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">On Sat, Jan 9, 2016 at 3:03 AM, Max Leske </span><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">wrote:</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class="">Hi,<br class=""><br class="">I have a rather annoying problem. I’m running a time critical piece of<br class="">code that reads a big (~90MB) image segment from a file. I’ve optimized<br class="">loading as far as possible and now GC takes far longer than the loading<br class="">itself (see the MessageTally output below).<br class="">I’m wondering if there’s any possibility to defer garbage collection<br class="">during the load.<br class=""><br class="">For completeness, here’s the use case: the process is socket activated,<br class="">which means that the first request coming in will start the process. When<br class="">the image starts it will load the segment to restore the last state of the<br class="">application and, once that’s done, serve the request. The critical time<br class="">includes vm startup, image startup, starting the server in the image and<br class="">loading the snapshot. With a big snapshot the loading time of the snapshot<br class="">is the most significant contributor.<br class=""><br class="">Maybe I could preallocate the needed memory to prevent the garbage<br class="">collector from running?<br class=""><br class="">I’d appreciate any ideas you have.<br class=""><br class=""><br class="">Cheers,<br class="">Max<br class=""><br class=""><br class="">PS: This needs to run on a Squeak 4.0.3 VM (no JIT)<br class=""><br class=""><br class=""><br class=""><br class="">Output from MessageTally:<br class=""><br class="">- 1624 tallies, 1624 msec.<br class=""><br class="">**Tree**<br class="">--------------------------------<br class="">Process: (40s) 592969728: nil<br class="">--------------------------------<br class="">4.4% {72ms} CBImageSegment class(NSImageSegment<br class="">class)>>basicSnapshot:from:do:<br class=""> 4.4% {72ms} CBImageSegment class(NSImageSegment<br class="">class)>>installSegmentFrom:andDo:<br class=""> 4.4% {72ms} CBImageSegment class(NSImageSegment<br class="">class)>>readSegmentFrom:<br class=""> 4.4% {72ms} NSSegmentStream>>readObject<br class=""> 4.4% {72ms} SmartRefStream>>nextAndClose<br class=""> 4.4% {72ms} SmartRefStream>>next<br class=""> 4.3% {70ms} SmartRefStream(ReferenceStream)>>next<br class=""> 4.3% {70ms} SmartRefStream(DataStream)>>next<br class=""> 3.2% {52ms}<br class="">NSImageSegment(ImageSegment)>>comeFullyUpOnReload:<br class=""> 3.2% {52ms} NSImageSegment>>restoreEndiannessAndRehash<br class="">**Leaves**<br class="">3.2% {52ms} NSImageSegment>>restoreEndiannessAndRehash<br class=""><br class="">**Memory**<br class=""> old +92,704,656 bytes<br class=""> young -8,008,252 bytes<br class=""> used +84,696,404 bytes<br class=""> free +1,287,768 bytes<br class=""><br class="">**GCs**<br class=""> full 2 totalling 954ms (59.0% uptime), avg<br class="">477.0ms<br class=""> incr 5 totalling 165ms (10.0% uptime), avg 33.0ms<br class=""> tenures 1 (avg 5 GCs/tenure)<br class=""> root table 0 overflows<br class=""></blockquote><br class="Apple-interchange-newline"></div></blockquote></div><br class=""></body></html>