[Vm-dev] VM Maker: VMMaker.oscog-cb.2376.mcz
Clément Bera
bera.clement at gmail.com
Thu Apr 26 12:17:07 UTC 2018
Eliot, if you have any idea why there's no look-up in compactor classes and
how to fix it... I'm pretty sure it's just a class-side method
somewhere... So I can remove all this duplication.
Anyway, I compiled VMs with PigCompactor, PlanningCompactor,
SelectiveCompactor & Sweep and they all seemed to work.
On Thu, Apr 26, 2018 at 2:12 PM, <commits at source.squeak.org> wrote:
>
> ClementBera uploaded a new version of VMMaker to project VM Maker:
> http://source.squeak.org/VMMaker/VMMaker.oscog-cb.2376.mcz
>
> ==================== Summary ====================
>
> Name: VMMaker.oscog-cb.2376
> Author: cb
> Time: 26 April 2018, 2:12:30.237019 pm
> UUID: bf5a9a98-571a-4a0e-985e-9e734854535a
> Ancestors: VMMaker.oscog-cb.2375
>
> Finally figured out that Slang is not able to do look-ups in compactor
> classes for some reason, hence only planning compactor was the only one
> compiling correctly since it defined everything inside...
>
> So I duplicated printTheBogon and postSwizzleAction to all subclasses.
>
> So I removed inheritance of SpurSelectiveCompactor to SpurSweeper, and
> duplicated used method.
>
> Eliot, if you have any idea why there's no look-up and how to fix it...
> I'm pretty sure it's just a class-side method somewhere... So I can remove
> all this duplication.
>
> =============== Diff against VMMaker.oscog-cb.2375 ===============
>
> Item was changed:
> CogClass subclass: #SpurCompactor
> instanceVariableNames: 'manager scavenger coInterpreter'
> classVariableNames: ''
> poolDictionaries: 'SpurMemoryManagementConstants VMBasicConstants
> VMSpurObjectRepresentationConstants'
> category: 'VMMaker-SpurMemoryManager'!
>
> + !SpurCompactor commentStamp: 'cb 4/26/2018 13:53' prior: 0!
> - !SpurCompactor commentStamp: 'cb 10/3/2017 10:49' prior: 0!
> Common superclass of all compactors to define apis and simulation
> variables.
>
> + IMPORTANT: This defines APIs only, subclassing is prohibited on
> auxiliary classes (Slang compiler won't find inherited methods).
> +
> The full GC in Spur is split in two, the marking phase and the compactor
> phase. The subclasses of SpurCompactor are implementations of the second
> phase, so they are called once the marking phase is finished. SpurCompactor
> is reponsible for:
> - freeing unmarked objects
> - compacting the live old space objects (though each subclass define
> what it does, some spurCompactor may not compact)
> - unmarking all objects remaining live
> - updating oops directly referred by the VM when they are moved
> (remapObj:/shouldRemapObj: thingy)
>
> The main apis are the following:
> - biasForGC/biasForSnapshot: tells the compactor if the GC is performed
> for snapshots or not, in general we want to compact more aggressively for
> snapshots to avoid saving large files with many unused space.
> - compact: main API, should free the unmarked object, unmark the objects
> remaining live and potentially compact the heap
> - remapObj:/shouldRemapObj: => Not really sure what this does, it seems
> it has to do with updating oops directly referred by the VM when they are
> moved.
>
>
> Instance Variables
> coInterpreter: <StackInterpreter>
> compactedCopySpace: <SpurNewSpaceSpace>
> manager:
> <SpurMemoryManager>!
>
> Item was changed:
> ----- Method: SpurCompactor>>postSwizzleAction (in category 'api') -----
> postSwizzleAction
> + self subclassResponsibility!
> - "Do nothing by default"!
>
> Item was changed:
> ----- Method: SpurCompactor>>printTheBogons: (in category 'debugging')
> -----
> printTheBogons: aBogon
> + self subclassResponsibility!
> - <inline: true>
> - coInterpreter
> - print: 'bogon '; printHexnp: aBogon; cr!
>
> Item was added:
> + ----- Method: SpurPigCompactor>>postSwizzleAction (in category
> 'compatibility') -----
> + postSwizzleAction
> + "do nothing"
> + !
>
> Item was added:
> + ----- Method: SpurPigCompactor>>printTheBogons: (in category 'debug
> printing') -----
> + printTheBogons: aBogon
> + <inline: true>
> + coInterpreter
> + print: 'bogon '; printHexnp: aBogon; cr!
>
> Item was added:
> + ----- Method: SpurPlanningCompactor>>postSwizzleAction (in category
> 'compatibility') -----
> + postSwizzleAction
> + "do nothing"
> + !
>
> Item was changed:
> + SpurCompactor subclass: #SpurSelectiveCompactor
> + instanceVariableNames: 'biasForGC segmentToFill'
> - SpurSweeper subclass: #SpurSelectiveCompactor
> - instanceVariableNames: 'segmentToFill'
> classVariableNames: 'MaxOccupationForCompaction'
> poolDictionaries: ''
> category: 'VMMaker-SpurMemoryManager'!
>
> + !SpurSelectiveCompactor commentStamp: 'cb 4/26/2018 13:59' prior: 0!
> - !SpurSelectiveCompactor commentStamp: 'cb 4/26/2018 12:53' prior: 0!
> SpurSelectiveCompactor compacts memory by selecting the memory segments
> with the most free space and compacting only those, to limit fragmentation
> while being really quick to perform. The algorithm is fast mostly because
> it does not update pointers: they are updated lazily during the next
> marking phase, so there is no need to read the fields of objects in other
> memory segments that the one compacted.
>
> The algorithm works as follow. First, a global sweep pass iterates over
> the memory linearly, changing unmarked objects to free chunks and
> concatenating free chunks. During the global sweep phase, the segments of
> the heap are analysed to determine the percentage of occupation. Second,
> the least occupied segments are compacted by copying the remaining live
> objects into an entirely free segment, called regionToFill (we detail later
> in the paragraph where regionToFill comes from), changing their values to
> forwarding objects and marking the free chunks as unavailable (removed from
> free list and marked as data objects). Third, the next marking phase
> removes all forwarders. Fourth, at the beginning of the next compaction
> phase the compacted segments from the previous GC can be entirely marked as
> free space (No need to check anything inside, there were only forwarders
> and trash data). One of the compacted segment is then selected as the
> segmentToFill, others are just marked as free chunks.
>
>
> The compaction is effectively partial, compacting only the most critical
> segments of the heap to limit fragmentation. Compaction time is crazy low,
> since a low number of objects are moved and pointer updated is lazily done
> during the next marking phase, while still preventing memory fragmentation.
>
> Now this works well when biasForGC is true, but when performing a
> snapshot, the compactor is just total crap (we need to figure out a
> solution).
>
> + IMPORTANT: I could not figure out to make inheritance work so I copied
> methods from SpurSweeper here.
> +
> segmentToFill <SegInfo> the segment that will be filled through the
> copying algorithm
>
> !
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>assertNoSegmentBeingCompacted (in
> category 'compaction') -----
> assertNoSegmentBeingCompacted
> -
> "Assertion only - no segment is being claimed at this point"
> + | segInfo |
> + <var: 'segInfo' type: #'SpurSegmentInfo *'>
> 0 to: manager numSegments - 1 do:
> + [:i|
> - [:i| | segInfo |
> segInfo := self addressOf: (manager segmentManager
> segments at: i).
> self deny: (self isSegmentBeingCompacted: segInfo)].
> !
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>biasForGC (in category 'api')
> -----
> + biasForGC
> + biasForGC := true.!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>biasForSnapshot (in category
> 'api') -----
> + biasForSnapshot
> + biasForGC := false.!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>bulkFreeChunkFrom: (in category
> 'sweep phase - copied from SpurSweeper') -----
> + bulkFreeChunkFrom: objOop
> + "ObjOop is either a freeChunk or an object to free, always in old
> space. The old space entity before objOop is necessarily a marked object.
> + Attempts to free as many byte from objOop, looking ahead for
> multiple freechunks / objects to free in a row"
> + | bytes start next currentObj |
> +
> + "Avoids pathological case, not point in dealing with non-mergeable
> free chunks, we would remove them and re-add them to the free list."
> + (self isSingleFreeObject: objOop) ifTrue: [^0].
> +
> + "We free unmarked objects and freechunks next to each others and
> merge them at the same time"
> + start := manager startOfObject: objOop.
> + currentObj := objOop.
> + bytes := 0.
> + [bytes := bytes + (manager bytesInObject: currentObj).
> + self freeEntity: currentObj.
> + next := manager objectStartingAt: start + bytes.
> + self canUseNextEntityAsFreeSpace: next]
> + whileTrue: [currentObj := next].
> +
> + manager addFreeChunkWithBytes: bytes at: start.
> +
> + ^ next!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>canUseAsFreeSpace: (in category
> 'sweep phase - copied from SpurSweeper') -----
> + canUseAsFreeSpace: objOop
> + <inline: true>
> + ^ (manager isFreeObject: objOop) or: [(manager isMarked: objOop)
> not]!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>canUseNextEntityAsFreeSpace: (in
> category 'sweep phase - copied from SpurSweeper') -----
> + canUseNextEntityAsFreeSpace: next
> + <inline: true>
> + ^ (manager oop: next isLessThan: manager endOfMemory) and: [self
> canUseAsFreeSpace: next]!
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>compactSegmentsToCompact (in
> category 'compaction') -----
> compactSegmentsToCompact
> "Forwards all objects in segments to compact and removes their
> freechunks"
> + | segInfo fillStart |
> + <var: 'segInfo' type: #'SpurSegmentInfo *'>
> - | fillStart |
> fillStart := segmentToFill segStart.
>
> "Removes initial free chunk in segment to fill... (Segment is
> entirely free)"
> manager detachFreeObject: (manager objectStartingAt: fillStart).
>
> "Compact each segment to compact..."
> 0 to: manager numSegments - 1 do:
> + [:i|
> - [:i| | segInfo |
> segInfo := self addressOf: (manager segmentManager
> segments at: i).
> (self isSegmentBeingCompacted: segInfo)
> ifTrue: [fillStart := self compactSegment: segInfo
> freeStart: fillStart ]].
>
> "Final free chunk in segment to fill..."
> manager
> addFreeChunkWithBytes: segmentToFill segSize - manager
> bridgeSize + segmentToFill segStart - fillStart
> at: fillStart.
>
> "Follow stack zone and caches..."
> self postForwardingAction
> !
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>computeSegmentsToCompact (in
> category 'compaction') -----
> computeSegmentsToCompact
> "Compute segments to claim: least occupied.
> Answers true if at least 1 segment is being compacted."
> -
> | canStillClaim aboutToClaim aboutToClaimSegment
> atLeastOneSegmentToCompact |
> + <var: 'aboutToClaimSegment' type: #'SpurSegmentInfo *'>
> atLeastOneSegmentToCompact := false.
> aboutToClaimSegment := self findNextSegmentToCompact.
> "Segment to fill is one of the segment compacted last GC.
> If no segment were compacted last GC, and that there is
> at least one segment to compact, allocate a new one."
> aboutToClaimSegment ifNil: [^false].
> segmentToFill ifNil: [self findOrAllocateSegmentToFill].
> canStillClaim := segmentToFill segSize - manager bridgeSize.
> [aboutToClaimSegment ifNil: [^atLeastOneSegmentToCompact].
> aboutToClaim := aboutToClaimSegment segSize - manager bridgeSize
> * ((self occupationOf: aboutToClaimSegment) + 1) // 255. "+1 to round up,
> this is approx"
> aboutToClaim < canStillClaim ] whileTrue:
> [self markSegmentAsBeingCompacted: aboutToClaimSegment.
> atLeastOneSegmentToCompact := true.
> canStillClaim := canStillClaim - aboutToClaim.
> aboutToClaimSegment := self findNextSegmentToCompact].
> ^atLeastOneSegmentToCompact!
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>findAndSetSegmentToFill (in
> category 'freeing') -----
> findAndSetSegmentToFill
> + | segInfo firstEntity |
> + <var: 'segInfo' type: #'SpurSegmentInfo *'>
> 0 to: manager numSegments - 1 do:
> + [:i|
> - [:i| | segInfo firstEntity |
> segInfo := self addressOf: (manager segmentManager
> segments at: i).
> firstEntity := manager objectStartingAt: segInfo segStart.
> ((manager isFreeObject: firstEntity) and: [(manager
> objectAfter: firstEntity limit: manager endOfMemory) = (manager
> segmentManager bridgeFor: segInfo)])
> ifTrue: [segmentToFill := segInfo. ^0]].
> !
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>findNextSegmentToCompact (in
> category 'compaction') -----
> findNextSegmentToCompact
> "Answers the next segment to compact or nil if none.
> The next segment to compact:
> - cannot be segment 0 (Segment 0 has specific objects
> (nil, true, etc.) and special size computed at start-up
> that we don't want to deal with)
> - cannot be be a segment already being compacted.
> - cannot contain pinned object (since we're in a copying GC)
> - cannot be entirely empty (no need to block that empty segment
> until next marking phase)
> - cannot have a high occupation rate (>
> MaxOccupationForCompaction)"
> + | leastOccupied leastOccupiedSegment tempOccupied segInfo |
> + <var: 'segInfo' type: #'SpurSegmentInfo *'>
> - | leastOccupied leastOccupiedSegment tempOccupied |
> leastOccupied := 255.
> 1 to: manager numSegments - 1 do:
> + [:i|
> - [:i| | segInfo |
> segInfo := self addressOf: (manager segmentManager
> segments at: i).
> ((self isSegmentBeingCompacted: segInfo) or: [segInfo
> containsPinned or: [manager segmentManager isEmptySegment: segInfo] ])
> ifFalse:
> [(tempOccupied := self occupationOf:
> segInfo) <= leastOccupied
> ifTrue: [ leastOccupied :=
> tempOccupied.
>
> leastOccupiedSegment := segInfo ]]].
> leastOccupied > MaxOccupationForCompaction ifTrue: [^nil].
> ^ leastOccupiedSegment!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>freeEntity: (in category 'sweep
> phase - copied from SpurSweeper') -----
> + freeEntity: entity
> + <inline: true>
> + (manager isFreeObject: entity)
> + ifFalse: "Freed old space objects are removed from
> remembered table"
> + [(manager isRemembered: entity) ifTrue:
> + [scavenger forgetObject: entity]]
> + ifTrue: "Merged old space free chunks are removed from
> free list"
> + [manager detachFreeObject: entity]
> + !
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>freePastSegmentsAndSetSegmentToFill
> (in category 'freeing') -----
> freePastSegmentsAndSetSegmentToFill
> "The first segment being claimed met becomes the segmentToFill.
> The others are just freed."
> + | segInfo |
> + <var: 'segInfo' type: #'SpurSegmentInfo *'>
> segmentToFill := nil.
> 0 to: manager numSegments - 1 do:
> + [:i|
> - [:i| | segInfo |
> segInfo := self addressOf: (manager segmentManager
> segments at: i).
> (self isSegmentBeingCompacted: segInfo)
> ifTrue:
> [self freeSegment: segInfo.
> segmentToFill ifNil: [segmentToFill :=
> segInfo]]]!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>isSingleFreeObject: (in category
> 'sweep phase - copied from SpurSweeper') -----
> + isSingleFreeObject: objOop
> + <inline: true>
> + | next |
> + ^ (manager isFreeObject: objOop) and:
> + [next := manager objectAfter: objOop limit: manager
> endOfMemory.
> + (manager oop: next isGreaterThanOrEqualTo: manager
> endOfMemory) or: [manager isMarked: next]]!
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>occupationOf: (in category
> 'segment access') -----
> + occupationOf: segInfo
> + <var: 'segInfo' type: #'SpurSegmentInfo *'>
> - occupationOf: segment
> - <var: 'segment' type: #'SpurSegmentInfo *'>
> "Swizzle is abused bit 8 isClaimed bits 0-7 occupation"
> + ^segInfo swizzle bitAnd: 16rFF!
> - ^segment swizzle bitAnd: 16rFF!
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>postSwizzleAction (in category
> 'api') -----
> postSwizzleAction
> "Since the compact abuses the swizzle field of segment, it needs
> to be rest after start-up."
> + | segInfo |
> + <var: 'segInfo' type: #'SpurSegmentInfo *'>
> 0 to: manager numSegments - 1 do:
> + [:i|
> - [:i| | segInfo |
> segInfo := self addressOf: (manager segmentManager
> segments at: i).
> segInfo swizzle: 0 ]!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>printTheBogons: (in category
> 'debugging') -----
> + printTheBogons: aBogon
> + <inline: true>
> + coInterpreter
> + print: 'bogon '; printHexnp: aBogon; cr!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>remapObj: (in category 'api')
> -----
> + remapObj: objOop
> + <api>
> + <inline: false>
> + ^manager vanillaRemapObj: objOop!
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>setOccupation:used:unused: (in
> category 'segment access') -----
> + setOccupation: segInfo used: used unused: unused
> + <var: 'segInfo' type: #'SpurSegmentInfo *'>
> - setOccupation: segment used: used unused: unused
> - <var: 'segment' type: #'SpurSegmentInfo *'>
> "Swizzle is abused bit 8 isClaimed bits 0-7 occupation
> Setting occupation resets the claim bit"
> | occupation |
> occupation := used * 255 // (used + unused).
> + segInfo swizzle: occupation!
> - segment swizzle: occupation!
>
> Item was changed:
> ----- Method: SpurSelectiveCompactor>>setOccupationAtIndex:used:unused:
> (in category 'segment access') -----
> setOccupationAtIndex: segmentIndex used: used unused: unused
> "Swizzle is abused bit 8 isClaimed bits 0-7 occupation
> Setting occupation resets the claim bit"
> | occupation segInfo |
> + <var: 'segInfo' type: #'SpurSegmentInfo *'>
> segInfo := self addressOf: (manager segmentManager segments at:
> segmentIndex).
> occupation := used * 255 // (used + unused).
> segInfo swizzle: occupation!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>shouldRemapObj: (in category
> 'api') -----
> + shouldRemapObj: objOop
> + <api>
> + ^manager vanillaShouldRemapObj: objOop!
>
> Item was added:
> + ----- Method: SpurSelectiveCompactor>>unmark: (in category 'sweep phase
> - copied from SpurSweeper') -----
> + unmark: objOop
> + self assert: ((manager isMarked: objOop) and: [(manager
> isFreeObject: objOop) not]).
> + (manager isSegmentBridge: objOop) ifFalse: [manager setIsMarkedOf:
> objOop to: false].
> + (manager isPinned: objOop) ifTrue: [manager segmentManager
> notePinned: objOop]!
>
> Item was added:
> + ----- Method: SpurSweeper>>postSwizzleAction (in category 'api') -----
> + postSwizzleAction
> + "do nothing"
> + !
>
> Item was added:
> + ----- Method: SpurSweeper>>printTheBogons: (in category 'debugging')
> -----
> + printTheBogons: aBogon
> + <inline: true>
> + coInterpreter
> + print: 'bogon '; printHexnp: aBogon; cr!
>
>
--
Clément Béra
https://clementbera.github.io/
https://clementbera.wordpress.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180426/22c86304/attachment-0001.html>
More information about the Vm-dev
mailing list