<div dir="ltr"><div><div>Ah great!<br></div>I had the same idea of instrumenting slang rather than analyzing the compiler warnings for the same purpose of filtering the noise out, but never took the time to do it...<br></div><br></div><div class="gmail_extra"><br><div class="gmail_quote">2017-01-25 18:22 GMT+01:00 Eliot Miranda <span dir="ltr"><<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br><div dir="ltr">Hi All,<div><br></div><div>    I had stalled in recent days on the new compactor.  The compactor appears to be working perfectly for me running on Mac OS, but when my colleague Bob Westergaard built a Newspeak VM for linux it crashed immediately (our Newspeak images invoke GC as soon as the low space watcher is spawned, because we're based on an older Squeak release).</div><div><br></div><div>Linux crashes whereas for me Mac OS doesn't crash for the combination of two reasons.  Linux places the heap much higher in the address space, typically creating heaps that extend into the up[per half of the address space.  The default type for oops in the VM is saint, which is signed.</div><div><br></div><div>So unless unsigned comparisons are used, oops in the upper half of the address space, which consequently have their sign bit set, will confuse enumerations, and hence bring the compactor crashing down.</div><div><br></div><div>The problem is how to find these unsigned comparisons.  I had taken the approach of simply finding test cases and running them in the debugger, but alas the compactor works perfectly in the simulator, because there's not enough address space to create a simulated heap > 2Gb, so the simulator works and the generated C doesn't work on Linux.  Running the two side by side trying to look for where they diverge was not productive given the difficulty of looking at a 47Mb image containing 875k objects.</div><div><br></div><div>And then the subconscious kicked in and I realized I could use Slang to analyze the parse tree and look for signed comparisons.  The first cut looked like this.  Yes, the open coding of building the code enrapture and running the type inference is ugly, but it's not that complicated.  The important bit at the bottom is</div><div><br></div><div><div><span style="white-space:pre-wrap">     </span>node isSend</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">        </span> and: [(#(< > <= >=) includes: node selector)</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">  </span> and: [({node receiver. node args first } anySatisfy:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                      </span>[:o| (cg typeFor: o in: m)</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                  </span>ifNil: [false]</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                      </span>ifNotNil: [:t| (cg isIntegralCType: t) and: [t first ~= $u]]])]]</div><div><br></div><div><div>SpurPlanningCompactor class>>#<wbr>identifySignedComparisons</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>"self identifySignedComparisons"</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>| vmm cg |</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>vmm := (VMMaker forPlatform: 'Cross')</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                               </span>interpreterClass: StackInterpreter;</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                         </span>options: #(ObjectMemory Spur32BitMemoryManager).</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">    </span>cg := [vmm buildCodeGeneratorForInterpret<wbr>er]</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                     </span>on: Notification</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                    </span>do: [:ex|</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                           </span>ex tag == #getVMMaker</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                       </span>ifTrue: [ex resume: vmm]</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                    </span>ifFalse: [ex pass]].</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">        </span>cg vmClass preGenerationHook: cg.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">   </span>cg inferTypesForImplicitlyTypedVa<wbr>riablesAndMethods.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">      </span>cg retainMethods: self selectors.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">   </span>cg prepareMethods.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>cg doInlining: true.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">        </span>self selectors do:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">          </span>[:sel|</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">              </span>(cg methodNamed: sel) ifNotNil:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                     </span>[:m|</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                        </span>m parseTree nodesDo:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                </span>[:node|</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                             </span>(node isSend</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                </span> and: [(#(< > <= >=) includes: node selector)</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                           </span> and: [{node receiver. node args first } anySatisfy:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                                </span>[:o| (cg typeFor: o in: m)</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                                          </span>ifNil: [false]</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                                              </span>ifNotNil: [:t| (cg isIntegralCType: t) and: [t first ~= $u]]]]]) ifTrue:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                    </span>[Transcript ensureCr; nextPutAll: sel; space; print: node; flush]]]]</div></div><div><br></div><div>Using the above was a doit to collect the results in a set allowed me to filter out the noise, and the final method looks like</div><div><br></div><div><div>identifySignedComparisons</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>"self identifySignedComparisons"</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>| vmm cg noise |</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">    </span>noise := #('(manager bytesInObject: largestFreeChunk) >= spaceEstimate'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                          </span>'(self classIndexOf: o*) > self isForwardedObjectClassIndexPun<wbr>'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                               </span>'GCModeFull > 0'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                         </span>'ReceiverIndex + (objectMemory integerValueOf: sp*) < (objectMemory lengthOf: o*)'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                               </span>'fmt* < manager firstCompiledMethodFormat'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                               </span>'fmt* < self firstCompiledMethodFormat'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                          </span>'fmt* <= 5'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                              </span>'gcPhaseInProgress > 0'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                          </span>'i <= finishIndex'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                               </span>'i >= 0'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                         </span>'numPointerSlots > 0'</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                            </span>'scavenger rememberedSetSize > 0').</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">      </span>vmm := (VMMaker forPlatform: 'Cross')</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                               </span>interpreterClass: StackInterpreter;</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                         </span>options: #(ObjectMemory Spur32BitMemoryManager).</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">    </span>cg := [vmm buildCodeGeneratorForInterpret<wbr>er]</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                     </span>on: Notification</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                    </span>do: [:ex|</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                           </span>ex tag == #getVMMaker</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                       </span>ifTrue: [ex resume: vmm]</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                    </span>ifFalse: [ex pass]].</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">        </span>cg vmClass preGenerationHook: cg.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">   </span>cg inferTypesForImplicitlyTypedVa<wbr>riablesAndMethods.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">      </span>cg retainMethods: self selectors.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">   </span>cg prepareMethods.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">  </span>cg doInlining: true.</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">        </span>self selectors do:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">          </span>[:sel|</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">              </span>(cg methodNamed: sel) ifNotNil:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                     </span>[:m|</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                        </span>m parseTree nodesDo:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                </span>[:node|</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                             </span>(node isSend</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                </span> and: [(#(< > <= >=) includes: node selector)</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                           </span> and: [({node receiver. node args first } anySatisfy:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                               </span>[:o| (cg typeFor: o in: m)</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                                          </span>ifNil: [false]</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                                              </span>ifNotNil: [:t| (cg isIntegralCType: t) and: [t first ~= $u]]])</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                              </span> and: [noise noneSatisfy: [:n| n match: node printString]]]]) ifTrue:</div><div><span class="m_-2874682713309964931gmail-Apple-tab-span" style="white-space:pre-wrap">                                       </span>[Transcript ensureCr; nextPutAll: sel; space; print: node; flush]]]]</div></div><div><br></div><div>and my output is</div><div><br></div><div><div>savedFirstFieldsSpaceInFreeChu<wbr>nk savedFirstFieldsSpace start >= nilObj</div><div>unmarkPinnedObjectsAndFindFirs<wbr>tUnpinnedOrFreeEntityFollowing<wbr>: (segments at: sweepIndex) segSize + (segments at: sweepIndex) segStart < nextObj</div><div>unmarkPinnedObjectsAndFindFirs<wbr>tUnpinnedOrFreeEntityFollowing<wbr>: nextObj >= manager endOfMemory</div><div>findNextMarkedPinnedAfter: nextObj >= manager endOfMemory</div><div>planCompactSavingForwarders toFinger <= (manager startOfObject: o)</div><div>planCompactSavingForwarders toFinger <= (manager startOfObject: previousPin)</div><div>updatePointers o2 >= firstFreeObject</div><div>updatePointers toFinger <= (manager startOfObject: o3)</div><div>updatePointers toFinger <= (manager startOfObject: previousPin)</div><div>savedFirstFieldsSpaceWasAlloca<wbr>ted savedFirstFieldsSpace start >= nilObj</div><div>freeFrom:upTo:previousPin: toFinger >= (segments at: i) segStart</div><div>freeFrom:upTo:previousPin: seg segSize + seg segStart < limit</div><div>freeFrom:upTo:previousPin: start := (self byteAt: pin + 7) = self numSlotsMask ifTrue: [pin - self baseHeaderSize] ifFalse: [pin] > toFinger</div><div>freeFrom:upTo:previousPin: (segments at: sweepIndex) segSize + (segments at: sweepIndex) segStart < nextObj1</div><div>freeFrom:upTo:previousPin: nextObj1 >= manager endOfMemory</div><div>freeFrom:upTo:previousPin: nextUnpinned >= limit</div><div>freeFrom:upTo:previousPin: nextObj >= manager endOfMemory</div><div>unmarkPinned: (segments at: sweepIndex) segSize + (segments at: sweepIndex) segStart < pinnedObj</div><div>compact savedFirstFieldsSpace start >= nilObj</div><div>compact savedFirstFieldsSpace start >= nilObj</div><div>updateSavedFirstFieldsSpaceIfN<wbr>ecessary savedFirstFieldsSpace start >= nilObj</div><div>endCompaction savedFirstFieldsSpace start >= nilObj</div><div>updatePointersInInitialImmobil<wbr>eObjects o >= firstFreeObject</div><div>copyAndUnmarkMobileObjects toFinger <= (manager startOfObject: o)</div><div>copyAndUnmarkMobileObjects toFinger <= (manager startOfObject: previousPin)</div><div>copyAndUnmarkMobileObjects bytes + 2 * 8 > availableSpace</div><div>copyAndUnmarkMobileObjects availableSpace > 0</div><div>releaseSavedFirstFieldsSpace savedFirstFieldsSpace start >= nilObj</div><div>updatePointersInMobileObjects toFinger <= (manager startOfObject: o)</div><div>updatePointersInMobileObjects toFinger <= (manager startOfObject: previousPin)</div></div><div><br></div><div><br></div><div>Less than half an hour's work.  Now I can squash those bugs :-)</div><div><br></div><div class="m_-2874682713309964931gmail_signature"><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div>
</div></div>
<br></blockquote></div><br></div>