<div dir="ltr">Ok I found the solution.<div><br></div><div>The problem was that <font face="monospace, monospace">SpurMemoryManager>>#ancilliaryClasses:</font> was only including <font face="monospace, monospace">self compactorClass</font> as <span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">ancilliary class. Including <font face="monospace, monospace">(self compactorClass withAllSuperclasses copyUpThrough: SpurCompactor) reverse</font> instead of <span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace">self compactorClass</font> allows to use inheritance. I can now write once <span style="font-family:monospace,monospace;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">postSwizzleAction </span><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="arial, helvetica, sans-serif">in</font></span><span style="font-family:monospace,monospace;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"> SpurCompactor</span><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="arial, helvetica, sans-serif"> and I don't have to override everything everywhere.</font></span></span></span></div><div><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="arial, helvetica, sans-serif"><br></font></span></span></span></div><div><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="arial, helvetica, sans-serif">Damned working Slang is a relation of love and hate, I love the simulator but right now I hate it.</font></span></span></span></div><div><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="arial, helvetica, sans-serif"><br></font></span></span></span></div><div><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="arial, helvetica, sans-serif"><br></font></span></span></span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 26, 2018 at 8:19 PM, Clément Bera <span dir="ltr"><<a href="mailto:bera.clement@gmail.com" target="_blank">bera.clement@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Thu, Apr 26, 2018 at 7:39 PM, Eliot Miranda <span dir="ltr"><<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="m_5485772114537546306gmail-">On Thu, Apr 26, 2018 at 9:50 AM, Clément Bera <span dir="ltr"><<a href="mailto:bera.clement@gmail.com" target="_blank">bera.clement@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">But I don't get it. In SpurManager we have subclasses and we can write methods in <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">SpurManager that can be used by all the subclasses. Cogit has multiple subclasses, and we can use any of the subclass re-using the superclass methods. StackInterpreterPrimitives re-use the methods from the superclass too. In each case we use one of the subclasses, but we re-use the superclass methods. Why can't we do that with the compactors? What makes it harder?</span></div></blockquote><div><br></div></span><div>The constraint is that one cannot have multiple implementations of a method not along the same inheritance path (unless marked with <doNotTranslate>; these are just forwarders). So one can use super (but the signatures must match perfectly), but one cannot have an implementation of foo in two different hierarchies, hence one cannot have a compact in SpurPlanningCompactor and in SpurSelectiveCompiler (one can, but these can't be included in the same Slang translation; one has to choose one or the other; look at methods in CCodeTranslator; it is just a dictionary from selector to TMethod). Unless, that is, we engineer something like I described below.</div></div></div></div></blockquote><div><br></div></span><div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I always use a single compactor at runtime.</span></div></div><div>I do not want to have multiple implementations of a method <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">along the same inheritance path.</span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I do not want an implementation in 2 different hierachy at runtime.</span></div><div><br></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">In SpurMemoryManager, there is <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">SpurMemoryManager>></span>globalGa<wbr>rbageCollect.</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="font-family:arial,helvetica,sans-serif">Spur32BitMemoryManager and </span><span style="font-family:arial,helvetica,sans-serif">Spur64BitMemoryManager do not override this method.</span><br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="arial, helvetica, sans-serif">When you select <span style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Spur32BitMemoryManager or <span style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Spur64BitMemoryManager as the memory manager, Slang compiler generate C code for <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">globalGarbageCollect.</span></span></span></font></div></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Now If I write:<div style="font-family:arial,sans-serif;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace">SpurCompactor>><wbr>postSwizzleAction</font></span></div><div style="font-family:arial,sans-serif;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace"> "do nothing"</font></span></div><div style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font><font face="arial, helvetica, sans-serif">and I do not override <span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">postSwizzleAction in SpurPlanningCompactor, when</span></font></font></span><font style="font-family:arial,helvetica,sans-serif"> I compile the VM selecting </font><span style="font-family:arial,helvetica,sans-serif">SpurPlanningCompacto<wbr>r as the runtime compactor, Slang compiler does not generate C code for postSwizzleAction and C compiler linker complains with: 'referenced symbol not implemented _</span><span style="font-family:arial,helvetica,sans-serif">postSwizzleAction".</span></div><div style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font><br class="m_5485772114537546306gmail-Apple-interchange-newline"></font></span></div><div style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font>So right now in the compactor classes I have to duplicate the method this way:</font></span></div></span></div><div><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><div style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><div style="font-family:arial,sans-serif;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace">SpurSweeper>>postSwizzleAction</font></span></div><div style="font-family:arial,sans-serif;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace"> "do nothing"</font></span></div><div style="font-family:arial,sans-serif;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace">SpurPlanningCompactor>><wbr>postSwizzleAction</font></span></div><div style="font-family:arial,sans-serif;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace"> "do nothing"</font></span></div><div style="font-family:arial,sans-serif;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace">SpurPigCompactor>><wbr>postSwizzleAction</font></span></div><div style="font-family:arial,sans-serif;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace"> "do nothing"</font></span></div><div style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="arial, helvetica, sans-serif">Since only <span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">SpurSelectiveCompactor overrides it.</span></font></span></div><div style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="arial, helvetica, sans-serif"><span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">And I have to do that with many methods.</span></font></span></div><div style="font-family:arial,sans-serif;color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div></span></span></div></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I don't understand why I cannot write code in SpurCompactor like it is done in SpurMemoryManager.</span></div><div><div class="h5"><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="m_5485772114537546306gmail-h5"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>I agree with your refactoring, right now I have multiple deadlines until May 18th, following deadlines are mid June, so I may have a few spare days and of May for pair programming.<div><div class="m_5485772114537546306gmail-m_-1427079196552267528h5"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 26, 2018 at 6:39 PM, Eliot Miranda <span dir="ltr"><<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Clément,<div class="gmail_extra"><br><div class="gmail_quote"><span>On Thu, Apr 26, 2018 at 8:51 AM, Clément Bera <span dir="ltr"><<a href="mailto:bera.clement@gmail.com" target="_blank">bera.clement@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">I fear there is a misunderstanding, I was not trying to have 2 compactors. I was just trying to have SelectiveCompactor subclassing Sweeper to reuse some methods instwad of duplicating them since SelectiveCompactor includes a sweep phase for non compacted segments and to compute segment occupation. That was not possible because some methods were not generated to C because they were in the superclass. I cannot even write a method in the common superclass SpurCompactor (for example defaulting postSwizzleAction to do nothing for all compactors but Selective) but I have to duplicate that method in all subclasses... </span></div></blockquote><div><br></div></span><div>There is no misunderstanding. I understand what you've written and I get why you want it. But I think we need two compactors, one for the two cases of programmer-initiated stop-the-world garbage collect (e.g. from the screen menu) and snapshot, and one for the case of the incremental mark-sweep GC (and our current implitic full GC when the scavenger detects that the heap has grown by the heapGrowthToSizeGCRatio since the last GC).</div><span><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="auto" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><br></div><div dir="auto" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">Currently at slang to c compilation time and in the simulator I change the compactor class setting to use different compactors. Having 2 compactors is very interesting, but I've dropped that idea for now since it is not that easy to implement. I wrote SelectiveCompactor as a research experiment, but since results look good (selective compaction time is 1/4th on average the compaction time of planning), we can for sure consider at some point to use it for real. Not sure if it is top priority though.</div></div></blockquote><div><br></div></span><div>It's tedious to rewrite the selectors so they don't conflict. But adding another variable to add an incrementalCompactor isn't that hard.</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="auto" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><div dir="auto"><br></div><div dir="auto">I need to build yet another compactor for my research which simulates G1 approach (remembered set per segment for inter segment references) to compare selective to it... I might not commit that one since it may pollute the code base with additionnal write barriers (I will see based on what I produce). </div></div></div></blockquote><div><br></div></span><div>I hope you do. The additional write barriers can be disabled in the other GCs can't they?</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="auto" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><div dir="auto"><br></div><div dir="auto">I feel the infrastructure is quite good right now for research evaluations on compactors. With the simulator it took me only 3 full days to implement SelectiveCompactor and 1 day for Sweeper. My current position is a research position and I am trying to spend part of my time using the Cog as a research framework. We will see how it works out.</div></div></div></blockquote><div><br></div></span><div>Nice!</div><div><br></div><div>There is a lot to discuss on restructuring the Slang output and/or bootstrapping the VM to be a pure Smalltalk VM. But this is expensive work, and it should be discussed in a different thread. But, just briefly, I wonder whether we could effect an inexpensive change if we have Slang</div><div>- identify the interfaces between the main classes/inst vars (coInterpreter, cogit, backEnd, methodZone, objectMemory, scavenger, compactor, segmentManager)</div><div>- distinguish between "trivial sends" that should still be inlined, and "major" sends that would remain in the interface</div><div>- map each of these objects to a simple struct and function-pointers C object representation (still statically, but allowing for limited polymorphism, because those that are polymorphic (say compactor) could be pointers-to-structs, not structs)</div><div>Then we could</div><div>- switch between compactors as you desire</div><div>- in Sista switch between a baseline JIT for unoptimized methods and a more aggressive JIT for optimized methods</div><div><br></div><div>I estimate this might take someone two to eight weeks, and a pair one to four weeks. I'd love to pair doing it.</div><div><div class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945h5"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="auto" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><div dir="auto"><span style="font-size:small">On Thu, Apr 26, 2018 at 4:46 PM, Eliot Miranda </span><span dir="ltr"><<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>></span><span style="font-size:small"> wrote:</span><br></div></div></div><div class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945m_8919785486407344669gmail-HOEnZb"><div class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945m_8919785486407344669gmail-h5"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <br><div dir="auto"><div><br></div><div><br>On Apr 26, 2018, at 7:41 AM, Eliot Miranda <<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div><div>Hi <span style="background-color:rgba(255,255,255,0)">Clément,</span><br></div><div><br>On Apr 26, 2018, at 5:34 AM, Clément Bera <<a href="mailto:bera.clement@gmail.com" target="_blank">bera.clement@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div><span></span></div></blockquote><blockquote type="cite"><div><div dir="ltr"><div>With this micro-bench:</div><div><div><font face="monospace, monospace">"alloc 320Mb"</font></div><div><font face="monospace, monospace">keeper1 := Array new: 9.</font></div><div><font face="monospace, monospace">keeper2 := Array new: 101.</font></div><div><font face="monospace, monospace">workspaceLocal := Array new: 80.</font></div><div><font face="monospace, monospace">1 to: 80 do: [:i | | a |</font></div><div><font face="monospace, monospace"><span style="white-space:pre-wrap"> </span>workspaceLocal at: i put: (a := Array new: 1000).</font></div><div><font face="monospace, monospace"><span style="white-space:pre-wrap"> </span>keeper1 at: i // 10 + 1 put: a.</font></div><div><font face="monospace, monospace"><span style="white-space:pre-wrap"> </span>1 to: 1000 do: [ :j | | w |</font></div><div><font face="monospace, monospace"><span style="white-space:pre-wrap"> </span>a at: j put: (w := WordArray new: 1000).</font></div><div><font face="monospace, monospace"><span style="white-space:pre-wrap"> </span>keeper2 at: i // 10 + 1 put: w ]].</font></div><div><font face="monospace, monospace">workspaceLocal := nil.</font></div><div><font face="monospace, monospace"><br></font></div><div><font face="monospace, monospace">Smalltalk garbageCollect.</font></div><div><font face="monospace, monospace">Smalltalk garbageCollect.</font></div><div><font face="monospace, monospace">Smalltalk garbageCollect.</font></div><div><font face="monospace, monospace"><br></font></div><div><font face="monospace, monospace">{Smalltalk vmParameterAt: 8. </font></div><div><font face="monospace, monospace">Smalltalk vmParameterAt: 18}</font></div><div><br></div><div>Time spent in full GC is:</div><div>- planning compactor ~800ms</div><div>- selective compactor ~450ms</div><div><br></div><div>Time spent in compaction:</div><div>- planning compactor ~450ms</div><div>- selective compactor ~125ms</div><div><br></div><div>Obviously this is a GC stress micro-benchmark which does *not* prove much, it's likely one of the case where both compactors are the most different, and there are other things to consider (Selective compactor waste more memory since it takes him more full GC to shrink allocated segments for example).</div><div><br></div><div>On larger benchmarks it seems compaction time is higher than expected though, Selective compactor may be relevant even prior to incremental marking. But if we really want to go that way we need to figure out something for snapshots...</div></div></div></div></blockquote><div><br></div>I think we need both. I know that's difficult, but a stop-the-world GC and a snapshot need something like SpurPlanningCompactor which<div>- compacts all of memory completely (as much as possible given potential pinned objects)</div><div>- is reasonably efficient (compared to SpurPigCompactor, which lived up to its name)</div><div>But the incremental collector needs incremental compaction and <span style="background-color:rgba(255,255,255,0)">SpurSelectiveCompactor does that (& it's really exciting; thank you!).</span></div><div><br></div><div>Given that Slang does translation to C with no objects in the target C, it means writing things clumsily, or it means engineering some object support in Slang. For me, KISS implies living with the Slang limitation for now. So add an incrementalCollector inst var and come up with a static renaming to avoid clashes.</div><div><br></div><div>If and when we reengineer to bootstrap the vm properly we can revisit this, but right now I think the restriction is in the nature of the beast.<br></div></div></blockquote><div><br></div>And din e we probably do want to be able to use SpurSelectiveCompactor when GC'ing in response to growth after scavenge (i.e. implicitly) I would add a flag to <span style="background-color:rgba(255,255,255,0)">SpurPlanningCompactor and modify its compact method to test the flag and invoke SpurSelectiveCompactor if set. If SpurSelectiveCompactor>>#co<wbr>mpact is renamed to e.g. compactLeastDenseSegment then this is straight-forward.</span><div><br><blockquote type="cite"><div><div><div><br><blockquote type="cite"><div><div dir="ltr"><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 26, 2018 at 1:00 PM, Clément Bera <span dir="ltr"><<a href="mailto:bera.clement@gmail.com" target="_blank">bera.clement@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">For the curious folks, here's a description of <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">SpurSelectiveCompactor:</span><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><br></span></div><div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);float:none;display:inline;font-size:12.8px"><div style="font-style:italic">SpurSelectiveCompactor compacts memory by selecting the memory segments with the most free space and compacting only those, to limit fragmentation while being really quick to perform. The algorithm is fast mostly because it does not update pointers: they are updated lazily during the next marking phase, so there is no need to read the fields of objects in other memory segments that the one compacted.</div><div style="font-style:italic"><br></div><div style="font-style:italic">The algorithm works as follow. First, a global sweep pass iterates over the memory linearly, changing unmarked objects to free chunks and concatenating free chunks. During the global sweep phase, the segments of the heap are analysed to determine the percentage of occupation. Second, the least occupied segments are compacted by copying the remaining live objects into an entirely free segment, called regionToFill (we detail later in the paragraph where regionToFill comes from), changing their values to forwarding objects and marking the free chunks as unavailable (removed from free list and marked as data objects). Third, the next marking phase removes all forwarders. Fourth, at the beginning of the next compaction phase the compacted segments from the previous GC can be entirely marked as free space (No need to check anything inside, there were only forwarders and trash data). One of the compacted segment is then selected as the segmentToFill, others are just marked as free chunks. </div><div style="font-style:italic"><br></div><div style="font-style:italic">The compaction is effectively partial, compacting only the most critical segments of the heap to limit fragmentation. Compaction time is crazy low, since a low number of objects are moved and pointer updated is lazily done during the next marking phase, while still preventing memory fragmentation.</div><div style="font-style:italic"><br></div><div>Although it's relevant research-wise, we don't need SelectiveCompactor in the short term. Full GC pause time is currently due to the Stop the World Mark and Compact algorithm. Mark pause is longer and we need to implement a tri-color incremental marking algorithm to solve this problem. Once done, compaction time becomes the biggest pause, which can still be a problem. SelectiveCompactor is a solution to decrease the compaction pause (SelectiveCompaction effectively does a sweep, which is very fast, and partial compaction without pointer update). The runtime may be a little slowed down due to the presence of more forwarders. Marking time is a little bit longer since it needs to remove more forwarders, though, as it is incremental there should not be longer pauses. Overall, the throughput might be a little lower (to confirm with benchmarks), but pauses are definitely smaller.</div><div><br></div></span></div></div><div class="gmail_extra"><div><div class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945m_8919785486407344669gmail-m_-3620193068297091477m_-9136847537164475202gmail-h5"><br><div class="gmail_quote">On Thu, Apr 26, 2018 at 11:12 AM, <span dir="ltr"><<a href="mailto:commits@source.squeak.org" target="_blank">commits@source.squeak.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <br>
ClementBera uploaded a new version of VMMaker to project VM Maker:<br>
<a href="http://source.squeak.org/VMMaker/VMMaker.oscog-cb.2373.mcz" rel="noreferrer" target="_blank">http://source.squeak.org/VMMak<wbr>er/VMMaker.oscog-cb.2373.mcz</a><br>
<br>
==================== Summary ====================<br>
<br>
Name: VMMaker.oscog-cb.2373<br>
Author: cb<br>
Time: 26 April 2018, 11:11:47.778949 am<br>
UUID: 9b389323-2181-4503-a361-d66ad8<wbr>7fa2de<br>
Ancestors: VMMaker.oscog-cb.2372<br>
<br>
Remove the APIs I added to iterate over free chunks (there was an existing API)<br>
<br>
Added assertValidFreeObject: to avoid stepping all the time in isValidFreeObject to know what's wrong. Obviously this new method cannot be used in the C code or we will have code in assertion-free VM, leading to the following pattern:<br>
self "Sorry stepping over isValidFreeObject all the time was killing me"<br>
cCode: [self assert: (self isValidFreeObject: child)]<br>
inSmalltalk: [self assertValidFreeObject: child].<br>
<br>
Since I now use forwarders in fullGC, adapted heap space integrity check.<br>
<br>
I was a little bit too aggressive in assertion in detachFreeObject: in last commit, reverted that.<br>
<br>
And SpurSelectiveCompactor is now working as an alternative compactor to Planning, Pig compactors and Sweeper! So exciting. Still needs some tuning for production use (Mostly snapshots consume high memory). SpurSelectiveCompactor compaction time is crazy low (almost as fast as a Sweep algorithm).<br>
<br>
I may write yet another compactor since I need to compare SelectiveCompactor with Garbage First multi-remembered table approach for research purpose...<br>
<br>
=============== Diff against VMMaker.oscog-cb.2372 ===============<br>
<br>
Item was removed:<br>
- ----- Method: SpurMemoryManager>>allOldSpace<wbr>FreeChunksDo: (in category 'object enumeration') -----<br>
- allOldSpaceFreeChunksDo: aBlock<br>
- <inline: true><br>
- <doNotGenerate> "Could be generated, but used for debug only"<br>
- self allOldSpaceFreeChunksFrom: self firstObject do: aBlock!<br>
<br>
Item was removed:<br>
- ----- Method: SpurMemoryManager>>allOldSpace<wbr>FreeChunksFrom:do: (in category 'object enumeration') -----<br>
- allOldSpaceFreeChunksFrom: initialObject do: aBlock<br>
- <inline: true><br>
- <doNotGenerate> "Could be generated, but used for debug only"<br>
- self allOldSpaceEntitiesFrom: initialObject<br>
- do: [:objOop|<br>
- (self isFreeObject: objOop) ifTrue:<br>
- [aBlock value: objOop]]!<br>
<br>
Item was changed:<br>
----- Method: SpurMemoryManager>>allocateOld<wbr>SpaceChunkOfExactlyBytes:suchT<wbr>hat: (in category 'free space') -----<br>
allocateOldSpaceChunkOfExactly<wbr>Bytes: chunkBytes suchThat: acceptanceBlock<br>
"Answer a chunk of oldSpace from the free lists that satisfies acceptanceBlock,<br>
if one of this size is available, otherwise answer nil. N.B. the chunk is simply a<br>
pointer, it has no valid header. The caller *must* fill in the header correctly."<br>
<var: #chunkBytes type: #usqInt><br>
| index node next prev child childBytes |<br>
<inline: true> "must inline for acceptanceBlock"<br>
"for debugging:" "totalFreeOldSpace := self totalFreeListBytes"<br>
<br>
index := chunkBytes / self allocationUnit.<br>
index < self numFreeLists ifTrue:<br>
[(freeListsMask anyMask: 1 << index) ifTrue:<br>
[(node := freeLists at: index) = 0<br>
ifTrue: [freeListsMask := freeListsMask - (1 << index)]<br>
ifFalse:<br>
[prev := 0.<br>
[node ~= 0] whileTrue:<br>
[self assert: node = (self startOfObject: node).<br>
self assert: (self isValidFreeObject: node).<br>
next := self fetchPointer: self freeChunkNextIndex ofFreeChunk: node.<br>
(acceptanceBlock value: node) ifTrue:<br>
[prev = 0<br>
ifTrue: [freeLists at: index put: next]<br>
ifFalse: [self storePointer: self freeChunkNextIndex ofFreeChunk: prev withValue: next].<br>
totalFreeOldSpace := totalFreeOldSpace - chunkBytes.<br>
^node].<br>
prev := node.<br>
node := next]]].<br>
^nil].<br>
<br>
"Large chunk. Search the large chunk list.<br>
Large chunk list organized as a tree, each node of which is a list of<br>
chunks of the same size. Beneath the node are smaller and larger<br>
blocks. When the search ends parent should hold the first chunk of<br>
the same size as chunkBytes, or 0 if none."<br>
node := 0.<br>
child := freeLists at: 0.<br>
[child ~= 0] whileTrue:<br>
+ [self "Sorry stepping over isValidFreeObject all the time was killing me"<br>
+ cCode: [self assert: (self isValidFreeObject: child)]<br>
+ inSmalltalk: [self assertValidFreeObject: child].<br>
- [self assert: (self isValidFreeObject: child).<br>
childBytes := self bytesInObject: child.<br>
childBytes = chunkBytes<br>
ifTrue: "size match; try to remove from list at node first."<br>
[node := child.<br>
[prev := node.<br>
node := self fetchPointer: self freeChunkNextIndex ofFreeChunk: node.<br>
node ~= 0] whileTrue:<br>
[(acceptanceBlock value: node) ifTrue:<br>
[self assert: (self isValidFreeObject: node).<br>
self storePointer: self freeChunkNextIndex<br>
ofFreeChunk: prev<br>
withValue: (self fetchPointer: self freeChunkNextIndex ofFreeChunk: node).<br>
totalFreeOldSpace := totalFreeOldSpace - chunkBytes.<br>
^self startOfObject: node]].<br>
(acceptanceBlock value: child) ifFalse:<br>
[^nil]. "node was right size but unaceptable."<br>
next := self fetchPointer: self freeChunkNextIndex ofFreeChunk: child.<br>
next = 0<br>
ifTrue: "no list; remove the interior node"<br>
[self unlinkSolitaryFreeTreeNode: child]<br>
ifFalse: "list; replace node with it"<br>
[self inFreeTreeReplace: child with: next].<br>
totalFreeOldSpace := totalFreeOldSpace - chunkBytes.<br>
^self startOfObject: child]<br>
ifFalse: "no size match; walk down the tree"<br>
[child := self fetchPointer: (childBytes < chunkBytes<br>
ifTrue: [self freeChunkLargerIndex]<br>
ifFalse: [self freeChunkSmallerIndex])<br>
ofFreeChunk: child]].<br>
^nil!<br>
<br>
Item was added:<br>
+ ----- Method: SpurMemoryManager>>assertValid<wbr>FreeObject: (in category 'free space') -----<br>
+ assertValidFreeObject: objOop<br>
+ <doNotGenerate> "If you want to generate this you want to use 'self assert: (self isValidFreeObject: objOop)' instead not to generate code in assertion-free VMs"<br>
+ | chunk |<br>
+ "duplicated assertions from isValidFreeObject: because I need to know what is wrong not only that it is not valid (I got bored of stepping inside isValidFreeObject:...)"<br>
+ self assert: (self oop: (self addressAfter: objOop) isLessThanOrEqualTo: endOfMemory).<br>
+ chunk := self fetchPointer: self freeChunkNextIndex ofFreeChunk: objOop.<br>
+ self assert: (chunk = 0 or: [self isFreeOop: chunk]).<br>
+ (self isLargeFreeObject: objOop) ifTrue: [<br>
+ "Tree assertions"<br>
+ chunk := self fetchPointer: self freeChunkParentIndex ofFreeChunk: objOop.<br>
+ self assert: (chunk = 0 or: [(self isFreeOop: chunk) and: [self isLargeFreeObject: chunk]]).<br>
+ chunk := self fetchPointer: self freeChunkSmallerIndex ofFreeChunk: objOop.<br>
+ self assert: (chunk = 0 or: [(self isFreeOop: chunk) and: [self isLargeFreeObject: chunk]]).<br>
+ chunk := self fetchPointer: self freeChunkLargerIndex ofFreeChunk: objOop.<br>
+ self assert: (chunk = 0 or: [(self isFreeOop: chunk) and: [self isLargeFreeObject: chunk]]). ]!<br>
<br>
Item was changed:<br>
----- Method: SpurMemoryManager>>checkHeapFr<wbr>eeSpaceIntegrity (in category 'debug support') -----<br>
checkHeapFreeSpaceIntegrity<br>
"Perform an integrity/leak check using the heapMap. Assume clearLeakMapAndMapAccessibleFr<wbr>eeSpace<br>
has set a bit at each free chunk's header. Scan all objects in the heap checking that no pointer points<br>
to a free chunk and that all free chunks that refer to others refer to marked chunks. Answer if all checks pass."<br>
| ok total |<br>
<inline: false><br>
<var: 'total' type: #usqInt><br>
ok := true.<br>
total := 0.<br>
0 to: self numFreeLists - 1 do:<br>
[:i|<br>
(freeLists at: i) ~= 0 ifTrue:<br>
[(heapMap heapMapAtWord: (self pointerForOop: (freeLists at: i))) = 0 ifTrue:<br>
[coInterpreter print: 'leak in free list '; printNum: i; print: ' to non-free '; printHex: (freeLists at: i); cr.<br>
self eek.<br>
ok := false]]].<br>
<br>
"Excuse the duplication but performance is at a premium and we avoid<br>
some tests by splitting the newSpace and oldSpace enumerations."<br>
self allNewSpaceEntitiesDo:<br>
[:obj| | fieldOop |<br>
(self isFreeObject: obj)<br>
ifTrue:<br>
[coInterpreter print: 'young object '; printHex: obj; print: ' is free'; cr.<br>
self eek.<br>
ok := false]<br>
ifFalse:<br>
[0 to: (self numPointerSlotsOf: obj) - 1 do:<br>
[:fi|<br>
fieldOop := self fetchPointer: fi ofObject: obj.<br>
(self isNonImmediate: fieldOop) ifTrue:<br>
[(heapMap heapMapAtWord: (self pointerForOop: fieldOop)) ~= 0 ifTrue:<br>
[coInterpreter print: 'object leak in '; printHex: obj; print: ' @ '; printNum: fi; print: ' = '; printHex: fieldOop; print: ' is free'; cr.<br>
self eek.<br>
ok := false]]]]].<br>
self allOldSpaceEntitiesDo:<br>
[:obj| | fieldOop |<br>
(self isFreeObject: obj)<br>
ifTrue:<br>
[(heapMap heapMapAtWord: (self pointerForOop: obj)) = 0 ifTrue:<br>
[coInterpreter print: 'leak in free chunk '; printHex: obj; print: ' is unmapped?!! '; cr.<br>
self eek.<br>
ok := false].<br>
fieldOop := self fetchPointer: self freeChunkNextIndex ofFreeChunk: obj.<br>
(fieldOop ~= 0<br>
and: [(heapMap heapMapAtWord: (self pointerForOop: fieldOop)) = 0]) ifTrue:<br>
[coInterpreter print: 'leak in free chunk '; printHex: obj; print: ' @ 0 = '; printHex: fieldOop; print: ' is unmapped'; cr.<br>
self eek.<br>
ok := false].<br>
(self isLargeFreeObject: obj) ifTrue:<br>
[self freeChunkParentIndex to: self freeChunkLargerIndex do:<br>
[:fi|<br>
fieldOop := self fetchPointer: fi ofFreeChunk: obj.<br>
(fieldOop ~= 0<br>
and: [(heapMap heapMapAtWord: (self pointerForOop: fieldOop)) = 0]) ifTrue:<br>
[coInterpreter print: 'leak in free chunk '; printHex: obj; print: ' @ '; printNum: fi; print: ' = '; printHex: fieldOop; print: ' is unmapped'; cr.<br>
self eek.<br>
ok := false]]].<br>
total := total + (self bytesInObject: obj)]<br>
ifFalse:<br>
[0 to: (self numPointerSlotsOf: obj) - 1 do:<br>
[:fi|<br>
+ (self isForwarded: obj)<br>
+ ifTrue: <br>
+ [self assert: fi = 0. "I'm now trying to use forwarders in GC algorithms..."<br>
+ fieldOop := self fetchPointer: fi ofMaybeForwardedObject: obj] <br>
+ ifFalse: "We keep #fetchPointer:ofObject: API here for assertions"<br>
+ [fieldOop := self fetchPointer: fi ofObject: obj].<br>
- fieldOop := self fetchPointer: fi ofObject: obj.<br>
(self isNonImmediate: fieldOop) ifTrue:<br>
[(heapMap heapMapAtWord: (self pointerForOop: fieldOop)) ~= 0 ifTrue:<br>
[coInterpreter print: 'object leak in '; printHex: obj; print: ' @ '; printNum: fi; print: ' = '; printHex: fieldOop; print: ' is free'; cr.<br>
self eek.<br>
ok := false]]]]].<br>
total ~= totalFreeOldSpace ifTrue:<br>
[coInterpreter print: 'incorrect totalFreeOldSpace; expected '; printNum: totalFreeOldSpace; print: ' found '; printNum: total; cr.<br>
self eek.<br>
ok := false].<br>
^ok!<br>
<br>
Item was changed:<br>
----- Method: SpurMemoryManager>>detachFreeO<wbr>bject: (in category 'free space') -----<br>
detachFreeObject: freeChunk<br>
"This is a rare operation, so its efficiency isn't critical.<br>
Having a valid prev link for tree nodes would help."<br>
<inline: false><br>
| chunkBytes result |<br>
chunkBytes := self bytesInObject: freeChunk.<br>
result := self allocateOldSpaceChunkOfExactly<wbr>Bytes: chunkBytes suchThat: [:f| f = freeChunk].<br>
self assert: result = (self startOfObject: freeChunk).<br>
- "Following is assertion only. Typical problem is that the free structures (tree/list) keep references to detached object somehow"<br>
- self cCode: '' inSmalltalk: <br>
- [self allOldSpaceFreeChunksDo: <br>
- [ :f | self assert: (self isValidFreeObject: f)]].<br>
!<br>
<br>
Item was changed:<br>
----- Method: SpurSelectiveCompactor>>compac<wbr>tSegmentsToCompact (in category 'compaction') -----<br>
compactSegmentsToCompact<br>
"Forwards all objects in segments to compact and removes their freechunks"<br>
| freeStart |<br>
freeStart := segmentToFill segStart.<br>
+ <br>
+ "Removes initial free chunk in segment to fill... (Segment is entirely free)"<br>
+ manager detachFreeObject: (manager objectStartingAt: freeStart).<br>
+ <br>
+ "Compact each segment to compact..."<br>
0 to: manager numSegments - 1 do:<br>
[:i| | segInfo |<br>
segInfo := self addressOf: (manager segmentManager segments at: i).<br>
(self isSegmentBeingCompacted: segInfo)<br>
ifTrue: [freeStart := self compactSegment: segInfo freeStart: freeStart ]].<br>
<br>
"Final free chunk in segment to fill..."<br>
manager <br>
addFreeChunkWithBytes: segmentToFill segSize - manager bridgeSize + segmentToFill segStart - freeStart <br>
at: freeStart.<br>
<br>
+ "Follow stack zone and caches..."<br>
self postForwardingAction<br>
!<br>
<br>
Item was changed:<br>
----- Method: SpurSelectiveCompactor>>findAn<wbr>dSetSegmentToFill (in category 'freeing') -----<br>
findAndSetSegmentToFill<br>
0 to: manager numSegments - 1 do:<br>
[:i| | segInfo firstEntity |<br>
segInfo := self addressOf: (manager segmentManager segments at: i).<br>
firstEntity := manager objectStartingAt: segInfo segStart.<br>
((manager isFreeObject: firstEntity) and: [(manager objectAfter: firstEntity limit: manager endOfMemory) = (manager segmentManager bridgeFor: segInfo)])<br>
+ ifTrue: [segmentToFill := segInfo. ^0]].<br>
- ifTrue: [segmentToFill := segInfo. manager detachFreeObject: firstEntity. ^0]].<br>
!<br>
<br>
Item was changed:<br>
----- Method: SpurSelectiveCompactor>>freePa<wbr>stSegmentsAndSetSegmentToFill (in category 'freeing') -----<br>
freePastSegmentsAndSetSegmentT<wbr>oFill <br>
+ "The first segment being claimed met becomes the segmentToFill. The others are just freed."<br>
- "The first segment being claimed met becomes the segmentToFill. The others are just freed"<br>
segmentToFill := nil.<br>
0 to: manager numSegments - 1 do:<br>
[:i| | segInfo |<br>
segInfo := self addressOf: (manager segmentManager segments at: i).<br>
(self isSegmentBeingCompacted: segInfo)<br>
ifTrue: <br>
+ [self freeSegment: segInfo.<br>
+ segmentToFill ifNil: [segmentToFill := segInfo]]]!<br>
- [segmentToFill<br>
- ifNil: [segmentToFill := segInfo]<br>
- ifNotNil: [self freeSegment: segInfo]]]!<br>
<br>
Item was changed:<br>
----- Method: SpurSelectiveCompactor>>select<wbr>iveCompaction (in category 'compaction') -----<br>
selectiveCompaction<br>
"Figures out which segments to compact and compact them into segmentToFill"<br>
| atLeastOneSegmentToCompact |<br>
self assertNoSegmentBeingCompacted.<br>
atLeastOneSegmentToCompact := self computeSegmentsToCompact.<br>
"If no compaction we don't pay forwarding cost (stack scan, cache scan, etc.)<br>
and we don't allocate segmentToFill if none available."<br>
atLeastOneSegmentToCompact <br>
ifTrue:<br>
[self assert: segmentToFill ~~ nil.<br>
+ self compactSegmentsToCompact].<br>
- self compactSegmentsToCompact]<br>
- ifFalse: <br>
- [segmentToFill ifNotNil: [self freeSegment: segmentToFill]].<br>
manager checkFreeSpace: GCModeFull.!<br>
<br>
Item was added:<br>
+ ----- Method: SpurSelectiveCompactorSimulato<wbr>r>>selectiveCompaction (in category 'compaction') -----<br>
+ selectiveCompaction<br>
+ super selectiveCompaction.<br>
+ manager allFreeObjectsDo: [:objOop | manager assertValidFreeObject: objOop]!<br>
<br>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945m_8919785486407344669gmail-m_-3620193068297091477m_-9136847537164475202gmail-HOEnZb"><font color="#888888">-- <br><div class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945m_8919785486407344669gmail-m_-3620193068297091477m_-9136847537164475202gmail-m_4548615913177328052gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><span style="font-size:12.8px">Clément Béra<br></span><span style="color:rgb(0,0,238)"><a href="https://clementbera.github.io/" target="_blank">https://clementbera.github.io/</a></span><div style="font-size:12.8px"><a href="https://clementbera.wordpress.com/" target="_blank">https://clementbera.wordpress.<wbr>com/</a></div></div></div></div></div></div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945m_8919785486407344669gmail-m_-3620193068297091477m_-9136847537164475202gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><span style="font-size:12.8px">Clément Béra<br></span><span style="color:rgb(0,0,238)"><a href="https://clementbera.github.io/" target="_blank">https://clementbera.github.io/</a></span><div style="font-size:12.8px"><a href="https://clementbera.wordpress.com/" target="_blank">https://clementbera.wordpress.<wbr>com/</a></div></div></div></div></div></div>
</div></div></div>
</div></blockquote></div></div></div></blockquote></div></div><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945m_8919785486407344669gmail-m_-3620193068297091477gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><span style="font-size:12.8px">Clément Béra<br></span><span style="color:rgb(0,0,238)"><a href="https://clementbera.github.io/" target="_blank">https://clementbera.github.io/</a></span><div style="font-size:12.8px"><a href="https://clementbera.wordpress.com/" target="_blank">https://clementbera.wordpress.<wbr>com/</a></div></div></div></div></div></div>
</div>
</div></div></blockquote></div></div></div><span class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945m_8919785486407344669gmail_signature"><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div>
</font></span></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_5485772114537546306gmail-m_-1427079196552267528m_-5612509478363450945gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><span style="font-size:12.8px">Clément Béra<br></span><span style="color:rgb(0,0,238)"><a href="https://clementbera.github.io/" target="_blank">https://clementbera.github.io/</a></span><div style="font-size:12.8px"><a href="https://clementbera.wordpress.com/" target="_blank">https://clementbera.wordpress.<wbr>com/</a></div></div></div></div></div></div>
</div></div></div></div></div>
</blockquote></div></div></div><span class="m_5485772114537546306gmail-HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div class="m_5485772114537546306gmail-m_-1427079196552267528gmail_signature"><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div>
</font></span></div></div>
</blockquote></div></div></div><div><div class="h5"><br><br clear="all"><div><br></div>-- <br><div class="m_5485772114537546306gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><span style="font-size:12.8px">Clément Béra<br></span><span style="color:rgb(0,0,238)"><a href="https://clementbera.github.io/" target="_blank">https://clementbera.github.io/</a></span><div style="font-size:12.8px"><a href="https://clementbera.wordpress.com/" target="_blank">https://clementbera.wordpress.<wbr>com/</a></div></div></div></div></div></div>
</div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><span style="font-size:12.8px">Clément Béra<br></span><span style="color:rgb(0,0,238)"><a href="https://clementbera.github.io/" target="_blank">https://clementbera.github.io/</a></span><div style="font-size:12.8px"><a href="https://clementbera.wordpress.com/" target="_blank">https://clementbera.wordpress.com/</a></div></div></div></div></div></div>
</div>