<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 12 Jan 2016, at 19:47, John McIntosh <<a href="mailto:johnmci@smalltalkconsulting.com" class="">johnmci@smalltalkconsulting.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Lots of questions. <div class="">I had a memory policy that would gather data, you can find it here</div><div class=""><a href="https://drive.google.com/file/d/0BzM4orHA3iGxd1JPcDFxZExYZE0/view?usp=sharing" class="">https://drive.google.com/file/d/0BzM4orHA3iGxd1JPcDFxZExYZE0/view?usp=sharing</a><br class=""></div><div class=""><br class=""></div><div class="">Also it sets up targets like trying to get a incremental GC to happen within N milliseconds etc by adjust parms. </div><div class="">If you search the archive for SophieMemoryPolicy or MemoryPolicy you'll find more discussion </div></div></div></blockquote><div><br class=""></div><div>Cool, thanks. I’ve already played around with that a bit. I didn’t see anything special in the stats yet but I’ll need some more time.</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, Jan 12, 2016 at 8:39 AM, Max Leske <span dir="ltr" class=""><<a href="mailto:maxleske@gmail.com" target="_blank" class="">maxleske@gmail.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class="">Hi John,<div class=""><br class=""></div><div class="">Thanks for the explanation.</div><div class=""><br class=""></div><div class="">As far as I understand, these settings won’t help in my particular case (I experimented with turning the bias on and off before allocating the ByteArray but didn’t see any influence). Still good to know about them.</div><div class=""><br class=""></div><div class="">I still have a few questions regarding garbage collection.</div><div class=""><br class=""></div><div class="">1. When I allocate a ByteArray, as Eliot suggested, like so:</div><div class=""><br class=""></div><div class="">Smalltalk vmParameterAt: 24 put: 150*1024*1024.</div><div class="">ByteArray new: 70*1024*1024.</div><div class=""><br class=""></div><div class="">then I see that the young space has grown a lot, as expected, and free space is low:</div><div class=""><br class=""></div><div class="">before allocation:</div><div class=""><span style="white-space:pre-wrap" class=""> </span>memory 57,032,988 bytes</div><div class=""><span style="white-space:pre-wrap" class=""> </span>old 44,259,392 bytes (77.60000000000001%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>young 5,824 bytes (0.0%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>used 44,265,216 bytes (77.60000000000001%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>free 12,767,772 bytes (22.400000000000002%)</div><div class=""><br class=""></div><div class="">after allocation:</div><div class=""><span style="white-space:pre-wrap" class=""> </span>memory 117,463,180 bytes</div><div class=""><span style="white-space:pre-wrap" class=""> </span>old 39,666,076 bytes (33.800000000000004%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>young 76,523,392 bytes (65.10000000000001%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>used 116,189,468 bytes (98.9%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>free 1,273,712 bytes (1.1%)</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">If I now force an incremental GC with #garbageCollectMost then then *nothing changes*, i.e. young space is still full and free space is low. What I would expect, since incremental GC, as I understand it, marks and sweeps young space, is that young space becomes small and free space becomes large (allocated memory of the ByteArray moves from young to free space):</div><div class=""><br class=""></div><div class="">after #garbageCollectMost:</div><div class=""><div class=""><span style="white-space:pre-wrap" class=""> </span>memory 117,463,180 bytes</div><div class=""><span style="white-space:pre-wrap" class=""> </span>old 39,666,076 bytes (33.800000000000004%</div><div class=""><span style="white-space:pre-wrap" class=""> </span> young 75,339,048 bytes (64.10000000000001%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>used 115,005,124 bytes (97.9%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>free 2,458,056 bytes (2.1%)</div></div><div class=""><br class=""></div><div class="">Only once I do a full garbageCollect does the ByteArray get collected:</div><div class=""><span style="white-space:pre-wrap" class=""> </span>memory 121,025,356 bytes</div><div class=""><span style="white-space:pre-wrap" class=""> </span>old 39,703,404 bytes (32.800000000000004%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>young 3,265,428 bytes (2.7%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>used 42,968,832 bytes (35.5%)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>free 78,056,524 bytes (64.5%)</div><div class=""><br class=""></div><div class="">Why do I see this behaviour? In Cog I see the expected outcome (free space big, young space small) even without triggering an incremental GC manually.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">2. There’s also still the question about where those 500ms disappear to (see last e-mail), which are unaccounted for in the MessageTally output.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Cheers,</div><div class="">Max</div><div class=""><div class="h5"><div class=""><br class=""></div><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On 11 Jan 2016, at 20:03, John McIntosh <<a href="mailto:johnmci@smalltalkconsulting.com" target="_blank" class="">johnmci@smalltalkconsulting.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class="">Ok as the author of <span style="font-size:13px" class="">setGCBiasToGrowGCLimit & </span>setGCBiasToGrow I was looking for a paper I gave at OOPSLA 2005 squeakfest on this. But can't find it. However the original note from January 18th 2005 explains<div class=""><br class=""></div><div class=""><span style="font-size:18px" class="">Mac 3.8.6b4</span></div><div class=""><span style="font-size:13px" class=""><br class=""></span></div><div class=""><span style="font-size:13px" class="">Smalltalk </span><span style="font-size:13px" class="">setGCBiasToGrowGCLimit</span><span style="font-size:13px" class="">: 16*1024*1024. "Set growth limit before full GC to 16MB"</span><br style="font-size:13px" class=""><span style="font-size:13px" class=""> Smalltalk setGCBiasToGrow: 1.</span><br style="font-size:13px" class=""><span style="font-size:13px" class=""> Set bias to grow upto GCLimit, this can avoid a problem where we attempt to avoid growing but results in thousands of incremental GC events as </span><span style="font-size:13px" class="">we approach a knee in a curve of space used versus the growth/compaction decision.</span><br class=""></div><div class=""><span style="font-size:13px" class=""><br class=""></span></div><div class=""><span style="font-size:13px" class="">Plus added this</span><br style="font-size:13px" class=""><span style="font-size:13px" class="">" A VM change will consider that after a tenure if the young space is less than 4MB then growth will happen to make young space greater than 4MB plus a calculated slack. Then after we've tenured N MB we will do a full GC, versus doing a full GC on every grow operation, this will trigger a shrink if required. For example we'll tenure at 75% and be bias to grow to 16MB before doing full GC."</span><br style="font-size:13px" class=""><br class=""><br style="font-size:13px" class=""><br style="font-size:13px" class=""><blockquote class="gmail_quote" style="font-size:13px;margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br class="">The Problem:<br class=""><br class="">Last weekend I built a new VM which has instrumentation to describe exactly what the GC is doing, also to<br class="">trigger a semaphore when an GC finishes, and to allow you to poke at more interesting things that control GC activity.<br class=""><br class="">What I found was an issue which we hadn't realized is there, well I'm sure people have seen it, but don't know why...<br class="">What happens is that as we are tenuring objects we are decreasing the young space from 4MB to Zero.<br class=""><br class="">Now as indicated in the table below if conditions are right (a couple of cases in the macrobenchmarks) why as you see the<br class="">number of objects we can allocate decreases to zero, and we actually don't tenure anymore once the survivors fall below 2000.<br class="">The rate at which young space GC activity occurs goes from say 8 per second towards 1000 per second, mind on fast machines<br class="">the young space ms accumulation count doesn't move much because the time taken to do this is under 1 millisecond, or 0, skewing<br class="">those statistics and hiding the GC time.<br class=""><br class="">AllocationCount Survivors<br class="">4000 5400<br class="">3209 3459<br class="">2269 2790<br class="">1760 1574<br class="">1592 2299<br class="">1105 1662<br class="">427 2355<br class="">392 2374<br class="">123 1472<br class="">89 1478<br class="">79 2<br class="">78 2<br class="">76 2<br class="">76 2<br class=""><br class="">Note how we allocate 76 objects, do a young space GC, then have two survivors, finally we reach the 200K minimum GC<br class="">threshold and do a full GC followed by growing young space. However this process is very painful. Also it's why the low space dialog<br class="">doesn't appear in a timely manner because we are attempting to approach the 200K limit and trying really hard by doing thousands of<br class="">young space GCed to avoid going over that limit. If conditions are right, then we get close but not close enough...<br class=""><br class="">What will change in the future.<br class=""><br class="">a) A GC monitoring class (new) will look at mark/sweep/Root table counts and decide when to do a tenure operation if iterating<br class="">over the root table objects takes too many iterations. A better solution would be to remember old objects and which slot has the young reference but that is harder to do.<br class=""><br class="">b) A VM change will consider that after a tenure if the young space is less than 4MB then growth will happen to make young space greater than 4MB plus a calculated slack. Then after we've tenured N MB we will do a full GC, versus doing a full GC on every grow operation, this will trigger a shrink if required. For example we'll tenure at 75% and be bias to grow to 16MB before doing full GC.<br class=""><br class="">c) To solve hitting the hard boundary when we can not allocate more space we need to rethink when the low semaphore is signaled and the rate of young space GC activity, signaling the semaphore earlier will allow a user to take action before things grind to a halt. I'm not quite sure how to do that yet.<br class=""></blockquote><div class=""><br class=""></div></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Mon, Jan 11, 2016 at 3:21 AM, Max Leske <span dir="ltr" class=""><<a href="mailto:maxleske@gmail.com" target="_blank" class="">maxleske@gmail.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br class=""><div style="word-wrap:break-word" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On 10 Jan 2016, at 20:22, <a href="mailto:vm-dev-request@lists.squeakfoundation.org" target="_blank" class="">vm-dev-request@lists.squeakfoundation.org</a> wrote:</div><br class=""><div class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">Hi Max,</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class=""> pre-Spur to avoid GC one has to a) grow memory by enough to do all the</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">processing you're going to do and b) change the shrinkage parameter so the</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">Vm won't shrink the heap back down before the processing is complete. To</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">do b) I suggest you modify setGCParameters. vm parameters 24 sets the</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">shrinkage threshold; see vmParameterAt:put:: "24 memory threshold above</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">whichto shrink object memory (read-write)". growMemory. Hmmm, I had</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">thoguht that there's a growMemoryBy: primitive in v3, but it appears there</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">isn't. So simply allocate a ByteArray of the desired size and then GC to</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">get rid of it. That should leave that much free space and then your load</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">should proceed without needing to GC.</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">Anyway, it's worth a try.</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""></div></blockquote><div class=""><br class=""></div><div class="">Thanks Eliot.</div><div class=""><br class=""></div><div class="">Setting the memory threshold helped. I’m still seeing one full GC which I’m trying to avoid. I’ve experimented with #setGCBiasToGrow: and #setGCBiasToGrowGCLimit: but I don’t fully understand what they do. </div><div class="">#setGCBiasToGrow: seems to turn memory growth on and off. But if this is turned off, how can the VM then allocate more memory?</div><div class="">#setGCBiasToGrowGCLimit: seems to control if the growth should trigger a full GC, which seems pretty much like what I need.</div><div class=""><br class=""></div><div class="">Unfortunately, while setting these options seems to have an influence, I can’t quite see the pattern, and that one full GC is still there. Maybe you could explain how these options work exactly?</div><div class=""><br class=""></div><div class="">One other question: the MessageTally output seems to be missing about 50% of the running time. Summing up over full GC + incremental GC + time spent in the tree, leaves about 500ms unaccounted for. Do you have any idea where that half second goes missing?</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Here’s the code I use experimentally:</div><div class=""><br class=""></div><div class=""><div class=""><div class="">MessageTally spyOn: [</div></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px" class=""><div class=""><div class=""><div class=""><div class="">| size shrinkThreshold |</div></div></div></div><div class=""><div class=""><div class=""><div class="">size := (self settings segmentsDirectory fileNamed: 'snapshot.bin') size.</div></div></div></div><div class=""><div class=""><div class=""><div class="">shrinkThreshold := (Smalltalk vmParameterAt: 24).</div></div></div></div><div class=""><div class=""><div class=""><div class="">Smalltalk vmParameterAt: 24 put: shrinkThreshold + (size*2). "(8MB + twice file size)"</div></div></div></div><div class=""><div class=""><div class=""><div class="">Smalltalk setGCBiasToGrowGCLimit: shrinkThreshold + (size*2).</div></div></div></div><div class=""><div class=""><div class=""><div class="">Smalltalk setGCBiasToGrow: 1. “enable growth??"</div></div></div></div><div class=""><div class=""><div class=""><div class="">ByteArray new: size*2.</div></div></div></div><div class=""><div class=""><div class=""><div class=""><br class=""></div><div class="">"incremental GC should take care of collecting the ByteArray, so I’m not doing anything</div></div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px" class=""><div class=""><div class=""><div class=""><div class="">manually here"</div></div></div></div></blockquote><div class=""><div class=""><div class=""><div class=""><br class=""></div></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px" class=""><div class=""><div class=""><div class=""><load snapshot> ].</div></div></div></blockquote><div class=""><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Cheers,</div><div class="">Max</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Output from current MessageTally:</div><div class=""><br class=""></div><div class=""><div class=""><div class=""> - 1123 tallies, 1125 msec.</div><div class=""><br class=""></div><div class="">**Tree**</div><div class="">--------------------------------</div><div class="">Process: (40s) 123994112: nil</div><div class="">--------------------------------</div><div class="">12.7% {143ms} CBImageSegment class(NSImageSegment class)>>basicSnapshot:from:do:</div><div class=""> 12.6% {141ms} CBImageSegment class(NSImageSegment class)>>installSegmentFrom:andDo:</div><div class=""> 12.6% {141ms} CBImageSegment class(NSImageSegment class)>>readSegmentFrom:</div><div class=""> 12.6% {141ms} NSSegmentStream>>readObject</div><div class=""> 12.6% {141ms} SmartRefStream>>nextAndClose</div><div class=""> 12.6% {141ms} SmartRefStream>>next</div><div class=""> 12.3% {138ms} SmartRefStream(ReferenceStream)>>next</div><div class=""> 12.3% {138ms} SmartRefStream(DataStream)>>next</div><div class=""> 10.6% {119ms} CBImageSegment(ImageSegment)>>comeFullyUpOnReload:</div><div class=""> |10.6% {119ms} CBImageSegment(NSImageSegment)>>restoreEndiannessAndRehash</div><div class=""> | 5.5% {62ms} Dictionary>>rehash</div><div class=""> | |2.8% {31ms} Dictionary>>associationsDo:</div><div class=""> | | |2.2% {25ms} Array(SequenceableCollection)>>do:</div><div class=""> | |1.7% {19ms} Dictionary>>noCheckAdd:</div><div class=""> | | 1.7% {19ms} Dictionary(HashedCollection)>>findElementOrNil:</div><div class=""> | | 1.2% {13ms} Dictionary>>scanFor:</div><div class=""> | 4.5% {51ms} primitives</div><div class=""> 1.2% {13ms} SmartRefStream(DataStream)>>readArray</div><div class=""> 1.2% {13ms} SmartRefStream>>next</div><div class=""> 1.2% {13ms} SmartRefStream(ReferenceStream)>>next</div><div class=""> 1.2% {13ms} SmartRefStream(DataStream)>>next</div><div class=""> </div><div class="">**Leaves**</div><div class=""><br class=""></div><div class="">**Memory**</div><div class=""><span style="white-space:pre-wrap" class=""> </span>old<span style="white-space:pre-wrap" class=""> </span>+94,031,228 bytes</div><div class=""><span style="white-space:pre-wrap" class=""> </span>young<span style="white-space:pre-wrap" class=""> </span>-9,207,660 bytes</div><div class=""><span style="white-space:pre-wrap" class=""> </span>used<span style="white-space:pre-wrap" class=""> </span>+84,823,568 bytes</div><div class=""><span style="white-space:pre-wrap" class=""> </span>free<span style="white-space:pre-wrap" class=""> </span>+90,024,824 bytes</div><div class=""><br class=""></div><div class="">**GCs**</div><div class=""><span style="white-space:pre-wrap" class=""> </span>full<span style="white-space:pre-wrap" class=""> </span>1 totalling 85ms (8.0% uptime), avg 85.0ms</div><div class=""><span style="white-space:pre-wrap" class=""> </span>incr<span style="white-space:pre-wrap" class=""> </span>15 totalling 271ms (24.0% uptime), avg 18.0ms</div><div class=""><span style="white-space:pre-wrap" class=""> </span>tenures<span style="white-space:pre-wrap" class=""> </span>10 (avg 1 GCs/tenure)</div><div class=""><span style="white-space:pre-wrap" class=""> </span>root table<span style="white-space:pre-wrap" class=""> </span>0 overflows</div><div class=""><br class=""></div></div></div><br class=""><blockquote type="cite" class=""><div class=""><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">On Sat, Jan 9, 2016 at 3:03 AM, Max Leske </span><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">wrote:</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote type="cite" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br class="">Hi,<br class=""><br class="">I have a rather annoying problem. I’m running a time critical piece of<br class="">code that reads a big (~90MB) image segment from a file. I’ve optimized<br class="">loading as far as possible and now GC takes far longer than the loading<br class="">itself (see the MessageTally output below).<br class="">I’m wondering if there’s any possibility to defer garbage collection<br class="">during the load.<br class=""><br class="">For completeness, here’s the use case: the process is socket activated,<br class="">which means that the first request coming in will start the process. When<br class="">the image starts it will load the segment to restore the last state of the<br class="">application and, once that’s done, serve the request. The critical time<br class="">includes vm startup, image startup, starting the server in the image and<br class="">loading the snapshot. With a big snapshot the loading time of the snapshot<br class="">is the most significant contributor.<br class=""><br class="">Maybe I could preallocate the needed memory to prevent the garbage<br class="">collector from running?<br class=""><br class="">I’d appreciate any ideas you have.<br class=""><br class=""><br class="">Cheers,<br class="">Max<br class=""><br class=""><br class="">PS: This needs to run on a Squeak 4.0.3 VM (no JIT)<br class=""><br class=""><br class=""><br class=""><br class="">Output from MessageTally:<br class=""><br class="">- 1624 tallies, 1624 msec.<br class=""><br class="">**Tree**<br class="">--------------------------------<br class="">Process: (40s) 592969728: nil<br class="">--------------------------------<br class="">4.4% {72ms} CBImageSegment class(NSImageSegment<br class="">class)>>basicSnapshot:from:do:<br class=""> 4.4% {72ms} CBImageSegment class(NSImageSegment<br class="">class)>>installSegmentFrom:andDo:<br class=""> 4.4% {72ms} CBImageSegment class(NSImageSegment<br class="">class)>>readSegmentFrom:<br class=""> 4.4% {72ms} NSSegmentStream>>readObject<br class=""> 4.4% {72ms} SmartRefStream>>nextAndClose<br class=""> 4.4% {72ms} SmartRefStream>>next<br class=""> 4.3% {70ms} SmartRefStream(ReferenceStream)>>next<br class=""> 4.3% {70ms} SmartRefStream(DataStream)>>next<br class=""> 3.2% {52ms}<br class="">NSImageSegment(ImageSegment)>>comeFullyUpOnReload:<br class=""> 3.2% {52ms} NSImageSegment>>restoreEndiannessAndRehash<br class="">**Leaves**<br class="">3.2% {52ms} NSImageSegment>>restoreEndiannessAndRehash<br class=""><br class="">**Memory**<br class=""> old +92,704,656 bytes<br class=""> young -8,008,252 bytes<br class=""> used +84,696,404 bytes<br class=""> free +1,287,768 bytes<br class=""><br class="">**GCs**<br class=""> full 2 totalling 954ms (59.0% uptime), avg<br class="">477.0ms<br class=""> incr 5 totalling 165ms (10.0% uptime), avg 33.0ms<br class=""> tenures 1 (avg 5 GCs/tenure)<br class=""> root table 0 overflows<br class=""></blockquote><br class=""></div></blockquote></div><br class=""></div><br class=""></blockquote></div><br class=""><br clear="all" class=""><div class=""><br class=""></div>-- <br class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class="">===========================================================================<br class="">John M. McIntosh. Corporate Smalltalk Consulting Ltd <a href="https://www.linkedin.com/in/smalltalk" target="_blank" class="">https://www.linkedin.com/in/smalltalk</a><br class="">===========================================================================<br class=""></div></div></div></div>
</div>
</div></blockquote></div><br class=""></div></div></div></div></blockquote></div><br class=""><br clear="all" class=""><div class=""><br class=""></div>-- <br class=""><div class="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class="">===========================================================================<br class="">John M. McIntosh. Corporate Smalltalk Consulting Ltd <a href="https://www.linkedin.com/in/smalltalk" target="_blank" class="">https://www.linkedin.com/in/smalltalk</a><br class="">===========================================================================<br class=""></div></div></div></div>
</div>
</div></blockquote></div><br class=""></body></html>