<div dir="ltr">Hi Guille,<div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 25, 2016 at 10:43 AM, Eliot Miranda <span dir="ltr">&lt;<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Guille,<div><br></div><div>    first, v important. can you put the image(s) you&#39;re working with somewhere I can download, and send me a URL?  I can be more helpful if I can run the tests too.</div><div><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 25, 2016 at 6:24 AM, Guille Polito <span dir="ltr">&lt;<a href="mailto:guillermopolito@gmail.com" target="_blank">guillermopolito@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> <br>
  
    
  
  <div text="#000000" bgcolor="#FFFFFF">
    Hi again!<br>
    <br>
    I snip most of the old thread to keep the relevant.<br>
    <br>
    <div>-------- Original Message --------<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr"><span class="">1 - The mourn table is being collected because it
        is not marked upon creation<br>
        </span><div class="gmail_extra">
          <div class="gmail_quote">
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF"> ===============<br>
                <br>
              </div>
            </blockquote><span class="">
            <div>I decided to move the assignment of marking out into
              markObjects:</div>
          </span></div>
        </div>
      </div>
    </blockquote>
    <br>
    Ok, it looks good for me, I did not know what would be a good place
    to put them without breaking something :)<br>
    <blockquote type="cite">
      <div dir="ltr">
        <div class="gmail_extra">
          <div class="gmail_quote"><span class="">
            <div><br>
            </div>
            <div><br>
            </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF">2 - Accesses to
                unscannedEphemerons are unaligned<br>
                ===============<br>
                <br>
              </div>
            </blockquote>
            </span><span class=""><div>Looks good.  I&#39;ve changed all the ephemeron methods to
              use bytesPerOop, cuz the ephemeron queue contains oops,
              but your fix is just as good.</div>
          </span></div>
        </div>
      </div>
    </blockquote>
    <br>
    cool! I thought so that would be the best.<br>
    <blockquote type="cite">
      <div dir="ltr">
        <div class="gmail_extra">
          <div class="gmail_quote">
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF"><span class=""> <br>
                <br>
                3 - While debugging I also had a DNU #intAt: in
                #primitiveIntegerAt<br>
                ==========================<br>
                <br></span><span class="">
                Question: besides reviewing, how should I commit these
                fixes? one issue per commit? all at once?</span></div>
            </blockquote><span class="">
            <div><br>
            </div>
            <div>Typically all at once.  What I try and do is commit a
              large set of related changes in a single commit, so I
              wouldn&#39;t mix the fixes to ephemerons with, say, fixes to
              time.  But equally I&#39;d commit any changes I made to help
              debugging in the same commit.  And all rules are made to
              be broken ;-).</div>
          </span></div>
        </div>
      </div>
    </blockquote>
    ok!<span class=""><br>
    <blockquote type="cite">
      <div dir="ltr">
        <div class="gmail_extra">
          <div class="gmail_quote">
            <div><br>
            </div>
            <div>Hey,  _thank you_!  This is really great and ephemerons
              won&#39;t be usable without testing like this.  I&#39;m in your
              debt.</div>
          </div>
        </div>
      </div>
    </blockquote>
    <br></span>
    Ha, there is no debt (or at least it would be in the opposite
    direction).<br>
    <br>
    Now I&#39;m working on another crash that I can easily reproduce using
    this script:<br>
    <br>
    Smalltalk supportsQueueingFinalization: true.<br>
    e := (1 to: 200000) collect: [ :i | Ephemeron key: (ObjectFinalizer
    receiver: &#39;test&#39;, &#39;asd&#39; selector: #logCr) value: Object new ].<br>
    Smalltalk garbageCollect.<br></div></blockquote></div></div></div></div></blockquote><div><br></div><div>Why the extra level of indirection?  The Ephemeron /is/ the object finaliser.  It can send finalise to its key.  Why bother with the extra level of wrapper?  It&#39;s wasteful; especially if we&#39;re attaching lots of ephemerons to things cuz we want to finalise something that has lots of instances.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">While debugging that in the simulator, we saw with clement that
    during #fireEphemeronsInRememberedSet,
    #fireEphemeronsOnEphemeronList we are always firing, tenuring and
    scavenging ephemerons regardless they are marked or not.<br>
    <br>
    fireEphemeronsInRememberedSet<br>
    [SNIP]<br>
             coInterpreter fireEphemeron: ephemeron.<br>
             manager<br>
                storePointerUnchecked: 0<br>
                ofObject: ephemeron<br>
                withValue: (self copyAndForward: (manager
    keyOfEphemeron: ephemeron)).<br>
            (self scavengeReferentsOf: ephemeron)<br>
    [SNIP]<br>
    <br>
    So I put an assertion to check that the key of the ephemeron is
    marked when scavenging and it is not always the case...<br>
             self assert: (manager isMarked: (manager keyOfEphemeron:
    ephemeron)).<br>
    <br>
    Is the code wrong? or I&#39;m missing something?<br></div></blockquote><div><br></div><div>The implementation of ephemerons in the scavenger and the implementation in mark-sweep are quite different.</div><div><br></div><div>Remember that what we&#39;re trying to do is find out if an object is only referenced from the transitive closure of ephemerons or not.  And the way that we do that in both cases is avoid processing ephemerons whose keys are not yet reachable from the roots until all objects reachable from the roots have been reached.  We do this by putting &quot;unscanned&quot; ephemerons in a queue, saving them until later.</div><div><br></div><div>In the scavenger the roots are the objects in the remembered set, the interpreter state (newMethod etc) and the stack zone, and GC is performed by copying all objects reachable from these roots in past and new spaces into future space, possibly tenuring overflowed objects into old space.  In the mark-sweep the roots are the specialObjectsArray, the interpreter state (newMethod etc) and the stack zone.</div><div><br></div><div>In the scavenger (a copying collector) this means that when we process the unscanned ephemerons their keys will either have been copied into future space or tenured to old space, in which case they were reachable from the roots, or they will not have been copied yet, in which case they are reachable only from ephemerons.  So in the scavenger marking is irrelevant; in fact /no/ objects should be marked when scavenging.  The important thing is whether a key is in past and new spaces or is in future and old spaces.</div><div><br></div><div>In the mark-sweep this means that when we process the unscanned ephemerons their keys will either be marked, in which case they were reachable from the roots, or unmarked, in which case they are reachable only from ephemerons. So the assert for marked-ness only makes sense in ephemeron processing in the mark-sweep collector.</div><div><br></div><div>HTH</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">Guille<div><div class="h5"><br>
    <br>
    <blockquote type="cite">
      <div dir="ltr">
        <div class="gmail_extra">
          <div class="gmail_quote">
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF">
                <div>
                  <div><br>
                    <br>
                    Guille<br>
                    <br>
                    <div>-------- Original Message --------<br>
                    </div>
                    <blockquote type="cite"> I see, I probably I need to
                      leave the simulator run for a longer time...
                      Because running the image using the assertion VM
                      took some time to crash, it may be the same case.<br>
                      <br>
                      <div>-------- Original Message --------<br>
                      </div>
                      <blockquote type="cite"> Some advance on my side,<br>
                        <br>
                        - Yes, apparently the image was already broken.
                        I started a new one and I can open it with the
                        simulator. It seems that as soon as I activate
                        Ephemerons and I have alive ephemerons the heap
                        corrupts (and since saving the image fires a
                        gc...).<br>
                        <br>
                        - Then, to catch the moment where the image
                        breaks, I prepared my image for debug with the
                        following script:<br>
                        <br>
                        Smalltalk snapshotPrimitive.<br>
                        f := ObjectFinalizer receiver: &#39;Hello&#39; selector:
                        #logCr.<br>
                        d := EphemeronDictionary new.<br>
                        d at: f put: f.<br>
                        f := nil.<br>
                        Smalltalk supportsQueueingFinalization: true.<br>
                        1 to: 2 do: [ :i | Smalltalk garbageCollect].<br>
                        <br>
                        Then I opened it with the simulator (latest
                        version: VMMaker.oscog-tfel.1864) and I had
                        several assertion failures in<br>
                        <br>
                        keyOfEphemeron: objOop<br>
                            &quot;Answer the object the ephemeron guards. 
                        This is its first element.&quot;<br>
                            self assert: ((self isNonImmediate: objOop)
                        and: [self isEphemeron: objOop]).<br>
                            ^self fetchPointer: 0 ofObject: objOop<br>
                        <br>
                        followedKeyOfEphemeron: objOop<br>
                            &quot;Answer the object the ephemeron guards. 
                        This is its first element.&quot;<br>
                            self assert: ((self isNonImmediate: objOop)
                        and: [self isEphemeron: objOop]).<br>
                            ^self followOopField: 0 ofObject: objOop<br>
                        <br>
                        So, I noticed the following:<br>
                        <br>
                        - Ephemerons in the unscanned list are marked,
                        fired, and the format of ephemerons is changed
                        to a non-ephemeron format<br>
                        - But they are not removed from the unscanned
                        list<br>
                        - then the mark #markAllUnscannedEphemerons
                        fails tons of assertions because it expects
                        ephemeron objects and the
                        (not-anymore-)ephemeron is still there in the
                        unscanned list<br>
                        <br>
                             self markInactiveEphemerons ifFalse:<br>
                                [self fireAllUnscannedEphemerons].<br>
                             self markAllUnscannedEphemerons<br>
                        <br>
                        The strange thing is that if I ignore the
                        assertions by proceeding, the thing continues
                        normally and it the image does not break...<br>
                        <br>
                        So I then compiled the VM, with the latest
                        version, and in trace mode to compare. I attach
                        the log of the stdout. But here there is a
                        résumé<br>
                        <br>
                        * First I see the same assertions I saw in the
                        simulator<br>
                        <br>
                        (isNonImmediate(objOop)) &amp;&amp;
                        (isEphemeron(objOop)) 41155<br>
                        <br>
                        (isNonImmediate(objOop)) &amp;&amp;
                        (isEphemeron(objOop)) 55477<br>
                        <br>
                        * Then there are tons of repetitions of the same
                        assertion:<br>
                        <br>
                        !(((GIV(mournQueue) != GIV(nilObj)) &amp;&amp;
                        (isonObjStack(anEphemeronOrWeakArray,
                        GIV(mournQueue))))) 60030<br>
                        <br>
                        Which I understand it means that we are adding
                        to the mourn queue all the time the same object.
                        Which is strange because it is a WeakArray with
                        a single slot which turns out to be nil (at
                        least at the moment I inspected it).<br>
                        <br>
                        * Then something breaks (!!) Looks like we are
                        trying to mark a forwarder. And then I have a
                        follow up of assertions which are probably
                        caused by the same thing.<br>
                        <br>
                        !(isForwarded(field)) 49489<br>
                        <br>
                        (classIndexOf(objToScan)) &gt;
                        (isForwardedObjectClassIndexPun()) 49537<br>
                        <br>
                        (classIndex &lt;= (tagMask())) || (classIndex
                        &gt;= (arrayClassIndexPun())) 49403<br>
                        <br>
                        objCouldBeClassObj(classObj) 49415<br>
                        <br>
                        addressCouldBeOop(objOop) 59997<br>
                        <br>
                        addressCouldBeObj(topOfObjStack((0 ==
                        (fetchPointerofObject(ObjStackTopx, objStack)) ?
                        fetchPointerofObject(ObjStackNextx, objStack) :
                        objStack))) 60002<br>
                        <br>
                        addressCouldBeOop(objOop) 59997<br>
                        <br>
                        * Finally there are some more <br>
                        <br>
                        !(isForwarded(field)) 49489<br>
                        <br>
                        until a segmentation fault arrives while
                        executing a fullGC<br>
                        <br>
                        ../results/pharo[0x80ceaa4]<br>
                        ../results/pharo[0x80cfde4]<br>
                        ../results/pharo(fullGC+0x9b)[0x80c95ea]<br>
                        ../results/pharo[0x80e2d49]<br>
                        ../results/pharo[0x80e5f4a]<br>
                        ../results/pharo[0x80ebca3]<br>
../results/pharo(ceStackOverflow+0xd7)[0x809356a]<br>
                        [0x91002c6]<br>
                        ../results/pharo(interpret+0x128)[0x8082b38]<br>
                        ../results/pharo(main+0x23a)[0x810a7db]<br>
/lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xf3)[0xf73b5a63]<br>
                        [0x1000]<br>
                        <br>
                        I&#39;ll continue digging a bit more,<br>
                        Guille<br>
                        <br>
                        <div>-------- Original Message --------<br>
                        </div>
                        <blockquote type="cite">
                          <div dir="ltr">Hi Guille,
                            <div class="gmail_extra"><br>
                              <div class="gmail_quote">On Thu, May 19,
                                2016 at 12:53 AM, Guille Polito <span dir="ltr">&lt;<a href="mailto:guillermopolito@gmail.com" target="_blank"></a><a href="mailto:guillermopolito@gmail.com" target="_blank">guillermopolito@gmail.com</a>&gt;</span>
                                wrote:<br>
                                <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                  <div><span> <br>
                                      <br>
                                      <div>-------- Original Message
                                        --------<br>
                                      </div>
                                      <blockquote type="cite">
                                        <div dir="ltr">Hi Guille,
                                          <div class="gmail_extra"><br>
                                            <div class="gmail_quote">On
                                              Wed, May 18, 2016 at 12:46
                                              AM, Guille Polito <span dir="ltr">&lt;<a href="mailto:guillermopolito@gmail.com" target="_blank"></a><a href="mailto:guillermopolito@gmail.com" target="_blank">guillermopolito@gmail.com</a>&gt;</span>
                                              wrote:<br>
                                              <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                                <div> Hello,<span><br>
                                                    <br>
                                                    <div>--------
                                                      Original Message
                                                      --------<br>
                                                    </div>
                                                    <blockquote type="cite">
                                                      <pre> </pre>
                                                      <br>
                                                      <fieldset></fieldset>
                                                      <br>
                                                      <div dir="ltr">Hi
                                                        Guille, Hi Pablo
                                                        (and welcome),<br>
                                                        <div class="gmail_extra"><br>
                                                          <div class="gmail_quote">On

                                                          Tue, May 17,
                                                          2016 at 8:37
                                                          AM, Guille
                                                          Polito <span dir="ltr">&lt;<a href="mailto:guillermopolito@gmail.com" target="_blank"></a><a href="mailto:guillermopolito@gmail.com" target="_blank">guillermopolito@gmail.com</a>&gt;</span>
                                                          wrote:<br>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
                                                          Hi Eliot, list<br>
                                                          <br>
                                                          I&#39;m working
                                                          here with
                                                          Pablo (Tesone)
                                                          on moving
                                                          forward the
                                                          Ephemeron
                                                          implementation.</blockquote>
                                                          </div>
                                                        </div>
                                                      </div>
                                                    </blockquote>
                                                  </span></div>
                                              </blockquote>
                                              <div><br>
                                              </div>
                                              <div><br>
                                              </div>
                                              <div>Where&#39;s &quot;here&quot;?   Are
                                                you in Lille?</div>
                                            </div>
                                          </div>
                                        </div>
                                      </blockquote>
                                    </span> Yup. Pablo is a new Phd
                                    student here :)<span><br>
                                      <blockquote type="cite">
                                        <div dir="ltr">
                                          <div class="gmail_extra">
                                            <div class="gmail_quote">
                                              <div><br>
                                              </div>
                                              <div><br>
                                              </div>
                                              <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                                <div><span>
                                                    <blockquote type="cite">
                                                      <div dir="ltr">
                                                        <div class="gmail_extra">
                                                          <div class="gmail_quote">
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                                          We first
                                                          installed
                                                          Eliot&#39;s
                                                          changeset,
                                                          added a #mourn
                                                          method and an
                                                          EphemeronDictionary

                                                          collection,
                                                          and then
                                                          started
                                                          testing
                                                          something like
                                                          this:<br>
                                                          <br>
                                                          f :=
                                                          ObjectFinalizer
                                                          receiver:
                                                          &#39;Hello&#39;
                                                          selector:
                                                          #logCr.<br>
                                                          d :=
                                                          EphemeronDictionary
                                                          new.<br>
                                                          <br>
                                                          d at: f put:
                                                          f.<br>
                                                          <br>
                                                          f := nil.<br>
                                                          Smalltalk
                                                          garbageCollect.<br>
                                                          </blockquote>
                                                          <div><br>
                                                          </div>
                                                          <div>So this
                                                          looks like
                                                          something
                                                          simulate
                                                          able.  Are you
                                                          able to use
                                                          the
                                                          simulator?  If
                                                          not, why not? 
                                                          <br>
                                                          </div>
                                                          </div>
                                                        </div>
                                                      </div>
                                                    </blockquote>
                                                    <br>
                                                  </span> For some
                                                  reason I have that
                                                  bytesToShift when
                                                  opening the image is
                                                  negative.<br>
                                                </div>
                                              </blockquote>
                                              <div><br>
                                              </div>
                                              <div>That is to be
                                                expected.  In the real
                                                VM the heap is located
                                                somewhere well above the
                                                bottom of the address
                                                space, typically above
                                                the program code.  In
                                                the simulator the heap
                                                is located either at 0
                                                (an interpreter or stack
                                                VM) or immediately above
                                                the code zone (in a
                                                Cogit VM).  So when an
                                                image that has been
                                                saved on the real VM is
                                                loaded into the
                                                simulator all oops have
                                                to be adjusted down and
                                                hence bytesToShift is
                                                negative.</div>
                                              <div><br>
                                              </div>
                                              <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                                <div> <br>
                                                      bytesToShift :=
                                                  objectMemory
                                                  memoryBaseForImageRead
                                                  - oldBaseAddr. 
                                                  &quot;adjust pointers for
                                                  zero base address&quot;<br>
                                                  <br>
                                                  So I cannot continue
                                                  loading because
                                                  addresses become
                                                  negative and I have
                                                  &quot;Improper Store into
                                                  indexable object kind
                                                  of errors&quot;.</div>
                                              </blockquote>
                                              <div><br>
                                              </div>
                                              <div>Can you post a back
                                                trace?  Where is this
                                                happening?  </div>
                                            </div>
                                          </div>
                                        </div>
                                      </blockquote>
                                    </span> I&#39;m working in latest pharo
                                    (5.0), using the same source code
                                    that is used in the Pharo CI jobs to
                                    build the VM (pharo branded). I
                                    would expect this branch to be
                                    stable, maybe it could be a bit
                                    behind your branch of development,
                                    but I think Esteban merges
                                    regularly.<br>
                                  </div>
                                </blockquote>
                                <div><br>
                                </div>
                                <div>Esteban is very busy trying to get
                                  UFFI working, and having just got
                                  Pharo 5 released and working on Pharo
                                  6 to be released.  I&#39;d prefer it if
                                  you used the tip of VMMaker.oscog. 
                                  Further, that&#39;s the package that
                                  Clément is using and he&#39;s your expert
                                  source of local help.</div>
                                <div><br>
                                </div>
                                <div> </div>
                                <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                  <div> <br>
                                    I&#39;m executing this,<br>
                                    <br>
                                    vm := StackInterpreterSimulator
                                    newWithOptions: #(#ObjectMemory
                                    #Spur32BitMemoryManager ).<br>
                                    vm desiredNumStackPages: 8. &quot;Makes
                                    simulation faster by creating fewer
                                    stack pages.&quot;<br>
                                    (vm openOn:
                                    &#39;/home/guille/Pharo/temp/ephemerons/broken.image&#39;)<br>
                                        openAsMorph; run.<br>
                                    <br>
                                    And here is the trace. The error
                                    happens while swizzling.<br>
                                    <br>
LittleEndianBitmap(Object)&gt;&gt;error:<br>
LittleEndianBitmap(Object)&gt;&gt;errorImproperStore<br>
LittleEndianBitmap(Object)&gt;&gt;at:put:<br>
Spur32BitMMLESimulator&gt;&gt;longAt:put:<br>
Spur32BitMMLESimulator(SpurMemoryManager)&gt;&gt;swizzleFieldsOfObject:<br>
Spur32BitMMLESimulator(SpurMemoryManager)&gt;&gt;adjustAllOopsBy:<br>
Spur32BitMMLESimulator(SpurMemoryManager)&gt;&gt;initializeObjectMemory:<br>
StackInterpreterSimulatorLSB(StackInterpreter)&gt;&gt;initializeInterpreter:<br>
                                    [ self initializeInterpreter:
                                    bytesToShift ] in
                                    StackInterpreterSimulatorLSB(StackInterpreterSimulator)&gt;&gt;openOn:extraMemory:





                                    in Block: [ self
                                    initializeInterpreter: bytesToShift
                                    ]<span><br>
                                    </span></div>
                                </blockquote>
                                <div><br>
                                </div>
                                <div> So the image is presumably already
                                  broken?  Looks to me like it was
                                  corrupted when saved.   Can you
                                  reproduce the problem that led to this
                                  image being broken from a working
                                  image?  </div>
                                <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                  <div><span>
                                      <blockquote type="cite">
                                        <div dir="ltr">
                                          <div class="gmail_extra">
                                            <div class="gmail_quote">
                                              <div>What version of
                                                VMMaker.oscog are you
                                                using?</div>
                                            </div>
                                          </div>
                                        </div>
                                      </blockquote>
                                    </span> Monticello reports
                                    VMMaker.oscog-eem.1855.<br>
                                    However, I&#39;m not sure I have the
                                    right number, because source code is
                                    now in git. <br>
                                  </div>
                                </blockquote>
                                <div><br>
                                </div>
                                <div>It would be so great if you could
                                  use Monticello, not git, and hence be
                                  able to contribute back.</div>
                                <div> </div>
                                <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                  <div><span>
                                      <blockquote type="cite">
                                        <div dir="ltr">
                                          <div class="gmail_extra">
                                            <div class="gmail_quote">
                                              <div>  Are you running in
                                                Pharo or Squeak?  If
                                                you&#39;re in Lille you
                                                could perhaps visit
                                                Clément&#39;s office and get
                                                him to take a look. 
                                                Clément, would that be
                                                ok?</div>
                                            </div>
                                          </div>
                                        </div>
                                      </blockquote>
                                    </span> I&#39;ll maybe see it with
                                    clement, but today I have to work on
                                    some other project...<span><br>
                                      <blockquote type="cite">
                                        <div dir="ltr">
                                          <div class="gmail_extra">
                                            <div class="gmail_quote">
                                              <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                                <div><span>
                                                    <blockquote type="cite">
                                                      <div dir="ltr">
                                                        <div class="gmail_extra">
                                                          <div class="gmail_quote">
                                                          <div>When
                                                          debugging the
                                                          VM there are
                                                          two main
                                                          levels of
                                                          support, one
                                                          is the
                                                          simulator
                                                          where there is
                                                          maximum
                                                          support for
                                                          debugging:</div>
                                                          <div>- asserts
                                                          on all the
                                                          time</div>
                                                          <div>-
                                                          arbitrary
                                                          breakpoints</div>
                                                          <div>-
                                                          attempting
                                                          every GC in a
                                                          copy of the
                                                          heap before
                                                          doing the real
                                                          GC so that
                                                          bugs in the GC
                                                          can be
                                                          investigated
                                                          without
                                                          needing to
                                                          construct a
                                                          reproducible
                                                          case after a
                                                          crash</div>
                                                          <div>- the
                                                          Smalltalk
                                                          environment to
                                                          inspect and
                                                          browse</div>
                                                          <div><br>
                                                          </div>
                                                          <div>The next
                                                          level is the
                                                          assert and
                                                          debug VMs.  If
                                                          you look in
                                                          the build
                                                          directories on
                                                          the Cog svn
                                                          branch you&#39;ll
                                                          see that all
                                                          of them build
                                                          three VMs, a
                                                          production VM
                                                          with maximum
                                                          optimisation
                                                          and asserts
                                                          excluded, an
                                                          assert VM with
                                                          -O1 and
                                                          asserts
                                                          enabled, and a
                                                          debug VM with
                                                          -O0 and
                                                          asserts
                                                          enabled.  So
                                                          if you either
                                                          don&#39;t see the
                                                          bug in the
                                                          simulator, or
                                                          the simulator
                                                          is too slow
                                                          for the case
                                                          being
                                                          examined, or
                                                          if the bug
                                                          doesn&#39;t show
                                                          up in the
                                                          simulator (the
                                                          worst of all
                                                          worlds), build
                                                          both assert
                                                          and debug VMs
                                                          and run with
                                                          the assert VM
                                                          first.</div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                        </div>
                                                      </div>
                                                    </blockquote>
                                                  </span> Well so far we
                                                  were using a VM
                                                  compiled for debug
                                                  with a graphical C
                                                  debugger. It was not
                                                  so bad. However, I
                                                  cannot say I&#39;m missing
                                                  a better debugger.</div>
                                              </blockquote>
                                              <div><br>
                                              </div>
                                              <div>&quot;Compiled for debug&quot;
                                                is vague.  Do you mean
                                                it is compiled with -g
                                                -O0, or in addition is
                                                compiled with -g -O0
                                                -DDEBUGVM=0 -DNDEBUG=1?</div>
                                              <div> </div>
                                            </div>
                                          </div>
                                        </div>
                                      </blockquote>
                                    </span> That would exactly be:<br>
                                    <br>
                                    { &#39;-g3&#39;. <br>
                                            &#39;-O0&#39;. <br>
                                            &#39;-msse2&#39;. <br>
                                            &#39;-D_GNU_SOURCE&#39;. <br>
                                            &#39;-DITIMER_HEARTBEAT=1&#39;. <br>
                                            &#39;-DNO_VM_PROFILE=1&#39;. <br>
                                            &#39;-DDEBUGVM=1&#39;.<br>
                                            &#39;-DNDEBUG&#39; }<br>
                                    <br>
                                    And Esteban told me to compile it
                                    with &#39;-UNDEBUG&#39; to get more
                                    information, but I did not try this
                                    yet.<br>
                                  </div>
                                </blockquote>
                                <div><br>
                                </div>
                                <div>Yes.  NDEBUG is what turns off
                                  asserts.  You don&#39;t have a proper
                                  assert or debug VM unless you do not
                                  use -NDEBUG.</div>
                                <div><br>
                                </div>
                                <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                  <div>Thanks!
                                    <div>
                                      <div><br>
                                        <blockquote type="cite">
                                          <div dir="ltr">
                                            <div class="gmail_extra">
                                              <div class="gmail_quote">
                                                <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                                  <div><span>
                                                      <blockquote type="cite">
                                                        <div dir="ltr">
                                                          <div class="gmail_extra">
                                                          <div class="gmail_quote">
                                                          <div>Note that
                                                          there is a
                                                          heap leak
                                                          checker which
                                                          can be enabled
                                                          both in the
                                                          simulator and
                                                          the assert and
                                                          debug VMs. 
                                                          See the
                                                          checkForLeaks
                                                          method and the
                                                          -leakcheck
                                                          argument.</div>
                                                          </div>
                                                          </div>
                                                        </div>
                                                      </blockquote>
                                                      <br>
                                                    </span> ok!<span><br>
                                                      <blockquote type="cite">
                                                        <div dir="ltr">
                                                          <div class="gmail_extra">
                                                          <div class="gmail_quote">
                                                          <div><br>
                                                          </div>
                                                          <div>Without
                                                          the simulator
                                                          or the assert
                                                          and debug VMs
                                                          you are flying
                                                          blind.  It is
                                                          /really/
                                                          productive to
                                                          use the
                                                          simulator for
                                                          debugging,
                                                          provided the
                                                          bug is
                                                          reproducible
                                                          within a short
                                                          amount of
                                                          time, as for
                                                          example your
                                                          case is above.</div>
                                                          </div>
                                                          </div>
                                                        </div>
                                                      </blockquote>
                                                      <br>
                                                    </span> Ok, gotcha!
                                                    By this afternoon
                                                    I&#39;ll have some news
                                                    probably.<br>
                                                    <br>
                                                    Thanks a lot!
                                                    <div>
                                                      <div><br>
                                                        <blockquote type="cite">
                                                          <div dir="ltr">
                                                          <div class="gmail_extra">
                                                          <div class="gmail_quote">
                                                          <div><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                                          <br>
                                                          <br>
                                                          However, as
                                                          soon as we
                                                          garbage
                                                          collect twice,
                                                          we have a VM
                                                          crash. We
                                                          started
                                                          debugging the
                                                          VM to see if
                                                          we could have
                                                          some more
                                                          clues.<br>
                                                          <br>
                                                          The first
                                                          thing we
                                                          noticed is
                                                          that the first
                                                          time the GC
                                                          runs, the
                                                          mournQueue is
                                                          nil. This is
                                                          of course
                                                          expected
                                                          because the
                                                          new
                                                          finalization
                                                          mechanism was
                                                          not active and
                                                          then there was
                                                          no need to
                                                          create the
                                                          mournQueue. We
                                                          saw that the
                                                          mournQueue is
                                                          actually
                                                          created in a
                                                          lazy fashion
                                                          when putting
                                                          queuing a
                                                          mourned object
                                                          (I refer
                                                          myself to
                                                          #queueMourner:
                                                          and
                                                          #ensureRoomOnObjStackAt:).
                                                          So the second
                                                          time the GC
                                                          passes, the
                                                          mournQueue is
                                                          there. So far
                                                          ok, but still
                                                          crashing.<br>
                                                          <br>
                                                          The crash
                                                          happens in the
                                                          call to<br>
                                                          <br>
                                                          markAndTraceObjStackandContents(GIV(mournQueue),




                                                          1);<br>
                                                          <br>
                                                          after the<br>
                                                          <br>
                                                              if
                                                          (!markAndTraceContents)
                                                          {<br>
                                                                 return;<br>
                                                              }<br>
                                                          <br>
                                                          But when
                                                          understanding
                                                          why, it starts
                                                          being less
                                                          clear to us
                                                          :). We used
                                                          the
                                                          printObjStack()
                                                          function and
                                                          we saw that:<br>
                                                          <br>
                                                          call
                                                          printObjStack(markStack)<br>
                                                          call
                                                          printObjStack(weaklingStack)<br>
                                                          <br>
                                                          and we saw in
                                                          the console
                                                          some output
                                                          that makes
                                                          sense.
                                                          However,
                                                          printing the
                                                          mournQueue in
                                                          the same
                                                          manner
                                                          produces some
                                                          strange output<br>
                                                          <br>
                                                          call
                                                          printObjStack(mournQueue)<br>
                                                          <br>
                                                          head 
                                                          0xb06e980 cx
                                                          18 (18) fmt 10
                                                          (10) sz 4092
                                                          (4092) myx:
                                                          4098 (4098)
                                                          unmkd<br>
                                                              topx: 14
                                                          next:       
                                                          0x0 free:     
                                                            0x0<br>
                                                          <br>
                                                          We noticed
                                                          that free and
                                                          next are 0x0
                                                          while the
                                                          others are
                                                          not...<br>
                                                          <br>
                                                          Finally we saw
                                                          there is
                                                          isValidObjStack(),
                                                          that gave us
                                                          the following
                                                          results:<br>
                                                          <br>
                                                          call
                                                          isValidObjStack(markStack)
                                                          =&gt; 1<br>
                                                          <br>
                                                          call
                                                          isValidObjStack(weaklingStack)
                                                          =&gt; 0<br>
                                                          p
                                                          objStackInvalidBecause
                                                          = &quot;marking but
                                                          page is
                                                          unmarked&quot;<br>
                                                          <br>
                                                          call
                                                          isValidObjStack(mournQueue)
                                                          =&gt; 0<br>
                                                          p
                                                          objStackInvalidBecause
                                                          = &quot;marking but
                                                          page is
                                                          unmarked&quot;<br>
                                                          <br>
                                                          <br>
                                                          So we assume
                                                          that the stack
                                                          creation is
                                                          wrong? We are
                                                          a bit lost in
                                                          here.<br>
                                                          <br>
                                                          Guille and
                                                          Pablo<br>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          <br clear="all">
                                                          <div><br>
                                                          </div>
                                                          -- <br>
                                                          <div>
                                                          <div dir="ltr">
                                                          <div><span style="font-size:small;border-collapse:separate">
                                                          <div>_,,,^..^,,,_<br>
                                                          </div>
                                                          <div>best, Eliot</div>
                                                          </span></div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                        </blockquote>
                                                        <br>
                                                      </div>
                                                    </div>
                                                  </div>
                                                </blockquote>
                                              </div>
                                              <br>
                                              <br clear="all">
                                              <div><br>
                                              </div>
                                              -- <br>
                                              <div>
                                                <div dir="ltr">
                                                  <div><span style="font-size:small;border-collapse:separate">
                                                      <div>_,,,^..^,,,_<br>
                                                      </div>
                                                      <div>best, Eliot</div>
                                                    </span></div>
                                                </div>
                                              </div>
                                            </div>
                                          </div>
                                        </blockquote>
                                        <br>
                                      </div>
                                    </div>
                                  </div>
                                </blockquote>
                              </div>
                              <br>
                              <br clear="all">
                              <div><br>
                              </div>
                              -- <br>
                              <div>
                                <div dir="ltr">
                                  <div><span style="font-size:small;border-collapse:separate">
                                      <div>_,,,^..^,,,_<br>
                                      </div>
                                      <div>best, Eliot</div>
                                    </span></div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </blockquote>
                        <br>
                      </blockquote>
                      <br>
                    </blockquote>
                    <br>
                  </div>
                </div>
              </div>
            </blockquote>
          </div>
          <br>
          <br clear="all">
          <div><br>
          </div>
          -- <br>
          <div>
            <div dir="ltr">
              <div><span style="font-size:small;border-collapse:separate">
                  <div>_,,,^..^,,,_<br>
                  </div>
                  <div>best, Eliot</div>
                </span></div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </div></div></div>

<br></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div>
</font></span></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div>
</div></div>