[Vm-dev] Sometimes it's too easy, part II

John McIntosh johnmci at smalltalkconsulting.com
Wed Jan 25 19:34:12 UTC 2017


I recall that the issue we had with making the 32bit version clean was a
for/while loop using a signed integer as pointer compare. If the start/end
of the object was either negative or positive it worked, but if the object
spanned the 2GB boundary then boom.

Using the start address in mmap was helpful to fiddle with where the image
started in memory.

On Wed, Jan 25, 2017 at 10:45 AM, Nicolas Cellier <
nicolas.cellier.aka.nice at gmail.com> wrote:

>
> Ah great!
> I had the same idea of instrumenting slang rather than analyzing the
> compiler warnings for the same purpose of filtering the noise out, but
> never took the time to do it...
>
>
> 2017-01-25 18:22 GMT+01:00 Eliot Miranda <eliot.miranda at gmail.com>:
>
>>
>> Hi All,
>>
>>     I had stalled in recent days on the new compactor.  The compactor
>> appears to be working perfectly for me running on Mac OS, but when my
>> colleague Bob Westergaard built a Newspeak VM for linux it crashed
>> immediately (our Newspeak images invoke GC as soon as the low space watcher
>> is spawned, because we're based on an older Squeak release).
>>
>> Linux crashes whereas for me Mac OS doesn't crash for the combination of
>> two reasons.  Linux places the heap much higher in the address space,
>> typically creating heaps that extend into the up[per half of the address
>> space.  The default type for oops in the VM is saint, which is signed.
>>
>> So unless unsigned comparisons are used, oops in the upper half of the
>> address space, which consequently have their sign bit set, will confuse
>> enumerations, and hence bring the compactor crashing down.
>>
>> The problem is how to find these unsigned comparisons.  I had taken the
>> approach of simply finding test cases and running them in the debugger, but
>> alas the compactor works perfectly in the simulator, because there's not
>> enough address space to create a simulated heap > 2Gb, so the simulator
>> works and the generated C doesn't work on Linux.  Running the two side by
>> side trying to look for where they diverge was not productive given the
>> difficulty of looking at a 47Mb image containing 875k objects.
>>
>> And then the subconscious kicked in and I realized I could use Slang to
>> analyze the parse tree and look for signed comparisons.  The first cut
>> looked like this.  Yes, the open coding of building the code enrapture and
>> running the type inference is ugly, but it's not that complicated.  The
>> important bit at the bottom is
>>
>> node isSend
>>  and: [(#(< > <= >=) includes: node selector)
>>  and: [({node receiver. node args first } anySatisfy:
>> [:o| (cg typeFor: o in: m)
>> ifNil: [false]
>> ifNotNil: [:t| (cg isIntegralCType: t) and: [t first ~= $u]]])]]
>>
>> SpurPlanningCompactor class>>#identifySignedComparisons
>> "self identifySignedComparisons"
>> | vmm cg |
>> vmm := (VMMaker forPlatform: 'Cross')
>> interpreterClass: StackInterpreter;
>> options: #(ObjectMemory Spur32BitMemoryManager).
>> cg := [vmm buildCodeGeneratorForInterpreter]
>> on: Notification
>> do: [:ex|
>> ex tag == #getVMMaker
>> ifTrue: [ex resume: vmm]
>> ifFalse: [ex pass]].
>> cg vmClass preGenerationHook: cg.
>> cg inferTypesForImplicitlyTypedVariablesAndMethods.
>> cg retainMethods: self selectors.
>> cg prepareMethods.
>> cg doInlining: true.
>> self selectors do:
>> [:sel|
>> (cg methodNamed: sel) ifNotNil:
>> [:m|
>> m parseTree nodesDo:
>> [:node|
>> (node isSend
>> and: [(#(< > <= >=) includes: node selector)
>> and: [{node receiver. node args first } anySatisfy:
>> [:o| (cg typeFor: o in: m)
>> ifNil: [false]
>> ifNotNil: [:t| (cg isIntegralCType: t) and: [t first ~= $u]]]]]) ifTrue:
>> [Transcript ensureCr; nextPutAll: sel; space; print: node; flush]]]]
>>
>> Using the above was a doit to collect the results in a set allowed me to
>> filter out the noise, and the final method looks like
>>
>> identifySignedComparisons
>> "self identifySignedComparisons"
>> | vmm cg noise |
>> noise := #('(manager bytesInObject: largestFreeChunk) >= spaceEstimate'
>> '(self classIndexOf: o*) > self isForwardedObjectClassIndexPun'
>> 'GCModeFull > 0'
>> 'ReceiverIndex + (objectMemory integerValueOf: sp*) < (objectMemory
>> lengthOf: o*)'
>> 'fmt* < manager firstCompiledMethodFormat'
>> 'fmt* < self firstCompiledMethodFormat'
>> 'fmt* <= 5'
>> 'gcPhaseInProgress > 0'
>> 'i <= finishIndex'
>> 'i >= 0'
>> 'numPointerSlots > 0'
>> 'scavenger rememberedSetSize > 0').
>> vmm := (VMMaker forPlatform: 'Cross')
>> interpreterClass: StackInterpreter;
>> options: #(ObjectMemory Spur32BitMemoryManager).
>> cg := [vmm buildCodeGeneratorForInterpreter]
>> on: Notification
>> do: [:ex|
>> ex tag == #getVMMaker
>> ifTrue: [ex resume: vmm]
>> ifFalse: [ex pass]].
>> cg vmClass preGenerationHook: cg.
>> cg inferTypesForImplicitlyTypedVariablesAndMethods.
>> cg retainMethods: self selectors.
>> cg prepareMethods.
>> cg doInlining: true.
>> self selectors do:
>> [:sel|
>> (cg methodNamed: sel) ifNotNil:
>> [:m|
>> m parseTree nodesDo:
>> [:node|
>> (node isSend
>> and: [(#(< > <= >=) includes: node selector)
>> and: [({node receiver. node args first } anySatisfy:
>> [:o| (cg typeFor: o in: m)
>> ifNil: [false]
>> ifNotNil: [:t| (cg isIntegralCType: t) and: [t first ~= $u]]])
>> and: [noise noneSatisfy: [:n| n match: node printString]]]]) ifTrue:
>> [Transcript ensureCr; nextPutAll: sel; space; print: node; flush]]]]
>>
>> and my output is
>>
>> savedFirstFieldsSpaceInFreeChunk savedFirstFieldsSpace start >= nilObj
>> unmarkPinnedObjectsAndFindFirstUnpinnedOrFreeEntityFollowing: (segments
>> at: sweepIndex) segSize + (segments at: sweepIndex) segStart < nextObj
>> unmarkPinnedObjectsAndFindFirstUnpinnedOrFreeEntityFollowing: nextObj >=
>> manager endOfMemory
>> findNextMarkedPinnedAfter: nextObj >= manager endOfMemory
>> planCompactSavingForwarders toFinger <= (manager startOfObject: o)
>> planCompactSavingForwarders toFinger <= (manager startOfObject:
>> previousPin)
>> updatePointers o2 >= firstFreeObject
>> updatePointers toFinger <= (manager startOfObject: o3)
>> updatePointers toFinger <= (manager startOfObject: previousPin)
>> savedFirstFieldsSpaceWasAllocated savedFirstFieldsSpace start >= nilObj
>> freeFrom:upTo:previousPin: toFinger >= (segments at: i) segStart
>> freeFrom:upTo:previousPin: seg segSize + seg segStart < limit
>> freeFrom:upTo:previousPin: start := (self byteAt: pin + 7) = self
>> numSlotsMask ifTrue: [pin - self baseHeaderSize] ifFalse: [pin] > toFinger
>> freeFrom:upTo:previousPin: (segments at: sweepIndex) segSize + (segments
>> at: sweepIndex) segStart < nextObj1
>> freeFrom:upTo:previousPin: nextObj1 >= manager endOfMemory
>> freeFrom:upTo:previousPin: nextUnpinned >= limit
>> freeFrom:upTo:previousPin: nextObj >= manager endOfMemory
>> unmarkPinned: (segments at: sweepIndex) segSize + (segments at:
>> sweepIndex) segStart < pinnedObj
>> compact savedFirstFieldsSpace start >= nilObj
>> compact savedFirstFieldsSpace start >= nilObj
>> updateSavedFirstFieldsSpaceIfNecessary savedFirstFieldsSpace start >=
>> nilObj
>> endCompaction savedFirstFieldsSpace start >= nilObj
>> updatePointersInInitialImmobileObjects o >= firstFreeObject
>> copyAndUnmarkMobileObjects toFinger <= (manager startOfObject: o)
>> copyAndUnmarkMobileObjects toFinger <= (manager startOfObject:
>> previousPin)
>> copyAndUnmarkMobileObjects bytes + 2 * 8 > availableSpace
>> copyAndUnmarkMobileObjects availableSpace > 0
>> releaseSavedFirstFieldsSpace savedFirstFieldsSpace start >= nilObj
>> updatePointersInMobileObjects toFinger <= (manager startOfObject: o)
>> updatePointersInMobileObjects toFinger <= (manager startOfObject:
>> previousPin)
>>
>>
>> Less than half an hour's work.  Now I can squash those bugs :-)
>>
>> _,,,^..^,,,_
>> best, Eliot
>>
>>
>
>


-- 
===========================================================================
John M. McIntosh. Corporate Smalltalk Consulting Ltd
https://www.linkedin.com/in/smalltalk
===========================================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20170125/a9b7be9a/attachment-0001.html>


More information about the Vm-dev mailing list