[Vm-dev] VM Maker: Cog-eem.328.mcz
commits at source.squeak.org
commits at source.squeak.org
Tue Aug 16 02:50:49 UTC 2016
Eliot Miranda uploaded a new version of Cog to project VM Maker:
http://source.squeak.org/VMMaker/Cog-eem.328.mcz
==================== Summary ====================
Name: Cog-eem.328
Author: eem
Time: 15 August 2016, 7:50:11.265382 pm
UUID: 09c34c1e-71c5-4a9f-870c-e441d6d24011
Ancestors: Cog-eem.327
Provide Spur32BitPreen to opreen 32-bit images, removing the excessive free space left by Spur's current poor compaction algorithm.
=============== Diff against Cog-eem.327 ===============
Item was added:
+ SimulatorHarness subclass: #Spur32BitPreen
+ instanceVariableNames: 'oldHeap newHeap map reverseMap oldInterpreter newInterpreter imageHeaderFlags savedWindowSize'
+ classVariableNames: ''
+ poolDictionaries: 'VMObjectIndices'
+ category: 'Cog-Bootstrapping'!
+
+ !Spur32BitPreen commentStamp: 'eem 8/15/2016 19:49' prior: 0!
+ A Spur32BitPreen is a simple image rewriter for 32-bit Spru images that eliminates free space and hence shrinks the preened image. Use via
+ Spur32BitPreen new preenImage: 'spur'
+ which will produce spur-preen.image and spur-preen.changes from spur.image and spur.changes.
+
+ Instance Variables
+ imageHeaderFlags: <Integer>
+ map: <Dictionary>
+ newHeap: <Spur32BitMMLESimulator>
+ newInterpreter: <StackInterpreterSimulatorLSB>
+ oldHeap: <Spur32BitMMLESimulator>
+ oldInterpreter: <StackInterpreterSimulatorLSB>
+ reverseMap: <Dictionary>
+ savedWindowSize: <Integer>
+
+ imageHeaderFlags
+ - flags word in image header
+
+ map
+ - map from oops in old image to oops in new image
+
+ newHeap
+ - the preened heap
+
+ newInterpreter
+ - the interpreter wrapping the preened heap
+
+ oldHeap
+ - the heap to be preened
+
+ oldInterpreter
+ - the interpreter wrapping the heap to be preened
+
+ reverseMap
+ - map from oops in new image to oops in old image
+
+ savedWindowSize
+ - screen size word in mage header
+ !
Item was added:
+ ----- Method: Spur32BitPreen>>clone: (in category 'bootstrap image') -----
+ clone: oldObj
+ | newObj hash |
+ newObj := newHeap
+ allocateSlots: (oldHeap numSlotsOf: oldObj)
+ format: (oldHeap formatOf: oldObj)
+ classIndex: (oldHeap classIndexOf: oldObj).
+ (hash := oldHeap rawHashBitsOf: oldObj) ~= 0 ifTrue:
+ [newHeap setHashBitsOf: newObj to: hash].
+ (oldHeap isImmutable: oldObj) ifTrue:
+ [newHeap setIsImmutableOf: newObj to: true].
+ (oldHeap isPinned: oldObj) ifTrue:
+ [newHeap setIsPinnedOf: newObj to: true].
+ self deny: (oldHeap isRemembered: oldObj).
+ self deny: (oldHeap isMarked: oldObj).
+ self deny: (oldHeap isGrey: oldObj).
+ reverseMap at: newObj put: oldObj.
+ ^map at: oldObj put: newObj!
Item was added:
+ ----- Method: Spur32BitPreen>>cloneFreeLists: (in category 'bootstrap image') -----
+ cloneFreeLists: oldObj
+ | newObj |
+ newObj := newHeap
+ allocateSlots: newHeap numFreeLists
+ format: newHeap wordIndexableFormat
+ classIndex: newHeap wordSizeClassIndexPun.
+ reverseMap at: newObj put: oldObj.
+ ^map at: oldObj put: newObj!
Item was added:
+ ----- Method: Spur32BitPreen>>cloneObjects (in category 'bootstrap image') -----
+ cloneObjects
+ "Clone all normal objects. Of hidden objects only clone the freeLists object and
+ the classTableRoot and class table pages. In particular, dont clone objStacks.
+ The refs to the objStacks are nilled out in fillInHeap."
+ | i freeListsObject |
+ i := 0.
+ freeListsObject := oldHeap freeListsObject.
+ oldHeap allOldSpaceObjectsDo:
+ [:obj32|
+ (i := i + 1) >= 100000 ifTrue:
+ [Transcript nextPut: $:; flush. i := 0].
+ obj32 = freeListsObject
+ ifTrue:
+ [self cloneFreeLists: obj32]
+ ifFalse:
+ [(self shouldClone: obj32) ifTrue:
+ [self clone: obj32]]]!
Item was added:
+ ----- Method: Spur32BitPreen>>fillInBitsObject:from: (in category 'bootstrap image') -----
+ fillInBitsObject: obj64 from: obj32
+ 0 to: (oldHeap numBytesOf: obj32) - 1 do:
+ [:i|
+ newHeap
+ storeByte: i
+ ofObject: obj64
+ withValue: (oldHeap fetchByte: i ofObject: obj32)]!
Item was added:
+ ----- Method: Spur32BitPreen>>fillInClassTable (in category 'bootstrap image') -----
+ fillInClassTable
+ self fillInPointerObject: (map at: oldHeap classTableRootObj) from: oldHeap classTableRootObj!
Item was added:
+ ----- Method: Spur32BitPreen>>fillInCompiledMethod:from: (in category 'bootstrap image') -----
+ fillInCompiledMethod: newObj from: oldObj
+ "oldInterpreter printOop: oldObj"
+ "newInterpreter printOop: newObj"
+ 0 to: (oldHeap numPointerSlotsOf: oldObj) - 1 do:
+ [:i| | oldOop newOop |
+ oldOop := oldHeap fetchPointer: i ofObject: oldObj.
+ newOop := self map32BitOop: oldOop.
+ newHeap
+ storePointerUnchecked: i
+ ofObject: newObj
+ withValue: newOop].
+ (oldInterpreter startPCOfMethod: oldObj)
+ to: (oldHeap numBytesOf: oldObj) - 1
+ do: [:j|
+ newHeap
+ storeByte: j
+ ofObject: newObj
+ withValue: (oldHeap fetchByte: j ofObject: oldObj)]!
Item was added:
+ ----- Method: Spur32BitPreen>>fillInHeap (in category 'bootstrap image') -----
+ fillInHeap
+ | heapEnd freeListsObj |
+ heapEnd := newHeap freeStart.
+ newHeap
+ nilObject: (map at: oldHeap nilObject);
+ falseObject: (map at: oldHeap falseObject);
+ trueObject: (map at: oldHeap trueObject);
+ specialObjectsOop: (map at: oldHeap specialObjectsOop);
+ lastHash: oldHeap lastHash;
+ setHiddenRootsObj: (map at: oldHeap classTableRootObj).
+ newHeap segmentManager
+ initSegmentForInImageCompilationFrom: newHeap nilObject
+ to: heapEnd + newHeap bridgeSize.
+ freeListsObj := newHeap objectAfter: newHeap trueObject.
+ "Nil-out the free lists."
+ newHeap
+ fillObj: freeListsObj numSlots: (newHeap numSlotsOf: freeListsObj) with: 0;
+ initializeFreeSpacePostLoad: freeListsObj;
+ initializePostBootstrap;
+ setEndOfMemory: (newHeap segmentManager bridgeAt: 0) + newHeap baseHeaderSize!
Item was added:
+ ----- Method: Spur32BitPreen>>fillInObjects (in category 'bootstrap image') -----
+ fillInObjects
+ "interpreter32 printOop: oldObj"
+ | i |
+ {oldHeap markStack. oldHeap weaklingStack. oldHeap mournQueue} do:
+ [:obj|
+ obj ~= oldHeap nilObject ifTrue:
+ [map at: obj put: (map at: oldHeap nilObject)]].
+ i := 0.
+ oldHeap allObjectsDo:
+ [:oldObj|
+ (i := i + 1) >= 10000 ifTrue:
+ [Transcript nextPut: $.; flush. i := 0].
+ (map at: oldObj ifAbsent: nil)
+ ifNotNil:
+ [:newObj| | format |
+ (newHeap numSlotsOf: newObj) > 0 ifTrue: "filter-out filtered objStack pages"
+ [format := oldHeap formatOf: oldObj.
+ (newHeap isPointersFormat: format)
+ ifTrue:
+ [self fillInPointerObject: newObj from: oldObj]
+ ifFalse:
+ [(newHeap isCompiledMethodFormat: format)
+ ifTrue: [self fillInCompiledMethod: newObj from: oldObj]
+ ifFalse: [self fillInBitsObject: newObj from: oldObj]]]]
+ ifNil: [self assert: (self isUnmappedObject: oldObj)]]!
Item was added:
+ ----- Method: Spur32BitPreen>>fillInPointerObject:from: (in category 'bootstrap image') -----
+ fillInPointerObject: newObj from: oldObj
+ 0 to: (oldHeap numSlotsOf: oldObj) - 1 do:
+ [:i|
+ newHeap
+ storePointerUnchecked: i
+ ofObject: newObj
+ withValue: (self map32BitOop: (oldHeap fetchPointer: i ofObject: oldObj))]!
Item was added:
+ ----- Method: Spur32BitPreen>>incrementPCField:ofObject:for: (in category 'bootstrap image') -----
+ incrementPCField: fieldIndex ofObject: obj64 for: method32
+ | value nLits |
+ value := newHeap fetchPointer: fieldIndex ofObject: obj64.
+ (newHeap isIntegerObject: value)
+ ifTrue:
+ [nLits := oldHeap literalCountOf: method32.
+ newHeap
+ storePointerUnchecked: fieldIndex
+ ofObject: obj64
+ withValue: (newHeap integerObjectOf: nLits + LiteralStart * 4 + (newHeap integerValueOf: value))]
+ ifFalse:
+ [self assert: (reverseMap at: value) = oldHeap nilObject]!
Item was added:
+ ----- Method: Spur32BitPreen>>initMaps (in category 'initialize-release') -----
+ initMaps
+ map := Dictionary new: oldHeap memory size // 32.
+ reverseMap := Dictionary new: oldHeap memory size // 32!
Item was added:
+ ----- Method: Spur32BitPreen>>isUnmappedObject: (in category 'bootstrap image') -----
+ isUnmappedObject: obj32
+ "Answer if obj32 is an object that is not cloned by the bootstrap."
+ ^obj32 = oldHeap freeListsObject
+ or: [(oldHeap isValidObjStackPage: obj32)]!
Item was added:
+ ----- Method: Spur32BitPreen>>map32BitOop: (in category 'bootstrap image') -----
+ map32BitOop: obj32
+ "oldInterpreter printOop: obj32"
+ ^map
+ at: obj32
+ ifAbsent:
+ [(oldHeap isImmediateCharacter: obj32)
+ ifTrue: [newHeap characterObjectOf: (oldHeap characterValueOf: obj32)]
+ ifFalse: [newHeap integerObjectOf: (oldHeap integerValueOf: obj32)]]!
Item was added:
+ ----- Method: Spur32BitPreen>>on: (in category 'public access') -----
+ on: imageName
+ (oldInterpreter := StackInterpreterSimulator newWithOptions: #(ObjectMemory Spur32BitMemoryManager))
+ openOn: imageName extraMemory: 0.
+ oldHeap := oldInterpreter objectMemory.
+ imageHeaderFlags := oldInterpreter getImageHeaderFlags.
+ savedWindowSize := oldInterpreter savedWindowSize.
+ newInterpreter := StackInterpreterSimulator newWithOptions: #(ObjectMemory Spur32BitMemoryManager).
+ newHeap := newInterpreter objectMemory.
+ newHeap
+ allocateMemoryOfSize: oldHeap oldSpaceSize
+ newSpaceSize: 8 * 1024 * 1024
+ stackSize: 16 * 1024
+ codeSize: 0.
+ newHeap bootstrapping: true.
+ self initMaps!
Item was added:
+ ----- Method: Spur32BitPreen>>preenImage (in category 'public access') -----
+ preenImage
+ self cloneObjects.
+ self fillInObjects.
+ self fillInClassTable.
+ self fillInHeap.
+ self reportSizes!
Item was added:
+ ----- Method: Spur32BitPreen>>preenImage: (in category 'public access') -----
+ preenImage: imageName
+ (Smalltalk classNamed: #FileReference) ifNotNil:
+ [^self preenImageUsingFileReference: imageName].
+ (Smalltalk classNamed: #FileDirectory) ifNotNil:
+ [^self preenImageUsingFileDirectory: imageName].
+ self error: 'at a loss as to what file system support to use'!
Item was added:
+ ----- Method: Spur32BitPreen>>preenImageUsingFileDirectory: (in category 'public access') -----
+ preenImageUsingFileDirectory: imageName
+ | dirName baseName dir |
+ dirName := FileDirectory dirPathFor: imageName.
+ baseName := (imageName endsWith: '.image')
+ ifTrue: [FileDirectory baseNameFor: imageName]
+ ifFalse: [FileDirectory localNameFor: imageName].
+ dir := dirName isEmpty ifTrue: [FileDirectory default] ifFalse: [FileDirectory default on: dirName].
+ self on: (dir fullNameFor: baseName, '.image').
+ [self preenImage]
+ on: Halt
+ do: [:ex|
+ "suppress halts from the usual suspects (development time halts)"
+ (#(fullGC compactImage) includes: ex signalerContext sender selector)
+ ifTrue: [ex resume]
+ ifFalse: [ex pass]].
+ self writeSnapshot: (dir fullNameFor: baseName, '-preen.image')
+ headerFlags: imageHeaderFlags
+ screenSize: savedWindowSize.
+ dir deleteFileNamed: baseName, '-preen.changes';
+ copyFileNamed: baseName, '.changes' toFileNamed: baseName, '-preen.changes'!
Item was added:
+ ----- Method: Spur32BitPreen>>reportSizes (in category 'bootstrap image') -----
+ reportSizes
+ | change oldHeapSize newHeapSize |
+ oldHeapSize := oldHeap oldSpaceSize.
+ newHeapSize := newHeap oldSpaceSize.
+ change := newHeapSize - oldHeapSize / oldHeapSize.
+ Transcript
+ nextPutAll: 'done.'; cr;
+ nextPutAll: 'old heap size: '; nextPutAll: oldHeapSize asStringWithCommas; tab;
+ nextPutAll: 'initial new heap size: '; nextPutAll: newHeapSize asStringWithCommas; cr;
+ nextPutAll: 'change: '; print: change * 100.0 maxDecimalPlaces: 2; nextPut: $%; cr;
+ flush.
+ newHeapSize := newHeap endOfMemory
+ - newHeap scavenger eden limit
+ - newHeap totalFreeListBytes.
+ change := newHeapSize - oldHeapSize / oldHeapSize.
+ Transcript
+ nextPutAll: 'final new heap size: '; nextPutAll: newHeapSize asStringWithCommas; tab;
+ nextPutAll: 'change: '; print: change * 100.0 maxDecimalPlaces: 2; nextPut: $%; cr;
+ flush!
Item was added:
+ ----- Method: Spur32BitPreen>>shouldClone: (in category 'bootstrap image') -----
+ shouldClone: obj32
+ ^(oldHeap isValidObjStackPage: obj32) not!
Item was added:
+ ----- Method: Spur32BitPreen>>writeSnapshot:headerFlags:screenSize: (in category 'snapshot') -----
+ writeSnapshot: imageFileName headerFlags: headerFlags screenSize: screenSizeInteger
+ newHeap
+ checkFreeSpace;
+ runLeakCheckerForFullGC.
+ newInterpreter
+ setImageHeaderFlagsFrom: headerFlags;
+ setDisplayForm: nil;
+ setSavedWindowSize: savedWindowSize;
+ imageName: imageFileName;
+ writeImageFileIO.
+ Transcript cr; show: 'Done!!'!
Item was changed:
----- Method: Spur32to64BitBootstrap>>cloneFreeLists: (in category 'bootstrap image') -----
cloneFreeLists: obj32
| obj64 |
obj64 := heap64
allocateSlots: heap64 numFreeLists
+ format: heap64 wordIndexableFormat
+ classIndex: heap64 wordSizeClassIndexPun.
- format: heap64 sixtyFourBitIndexableFormat
- classIndex: heap64 sixtyFourBitLongsClassIndexPun.
reverseMap at: obj64 put: obj32.
^map at: obj32 put: obj64!
More information about the Vm-dev
mailing list