[Vm-dev] VM Maker: VMMaker.oscog-eem.2984.mcz
commits at source.squeak.org
commits at source.squeak.org
Fri Jul 9 23:41:56 UTC 2021
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2984.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.2984
Author: eem
Time: 9 July 2021, 4:41:46.033726 pm
UUID: f8f3a253-0bf6-4dbd-b427-c815aefd1f89
Ancestors: VMMaker.oscog-eem.2983
Apple M1 ARMv8 good enough to recompile the Compiler package and update an 8 month old trunk image (October 29th 2020). The major weakness right now is that this uses the Apple cache flush code, not the JITted code, which as yet does nto work. I'll investigate soon.
Abstract setting codeModified & conditionally sending makeCodeZoneWritable/pthread_jit_write_protect_np into setCodeModified, invoked from the relevant places. Not all places can rely on flushICacheFrom:to: to lip back into executability. Do so explicitly in these places.
Simulation: simplify shortcutTrampoline:to: further, but hack in pushes & pops of link register using SPReg; pushWord:in: uses the NativeSPReg. Make linked send tracing work on ARMv8.
=============== Diff against VMMaker.oscog-eem.2983 ===============
Item was changed:
----- Method: CogARMv8Compiler class>>declareCVarsIn: (in category 'translation') -----
declareCVarsIn: aCCodeGenerator
#('hasAtomicInstructions' 'instructionCacheLineLength' 'instructionCacheFlushRequired' 'dataCacheLineLength' 'dataCacheFlushRequired') do:
[:varName|
aCCodeGenerator
declareVar: varName type: #'unsigned char';
+ removeConstant: varName capitalized].
+ aCCodeGenerator
+ var: #ceFlushDCache
+ declareC: '#if __APPLE__\static void (*ceFlushDCache)(usqIntptr_t from, usqIntptr_t to)\#endif'!
- removeConstant: varName capitalized]!
Item was added:
+ ----- Method: CogARMv8Compiler>>callFullInstructionByteSize (in category 'accessing') -----
+ callFullInstructionByteSize
+ ^8!
Item was changed:
----- Method: CogARMv8Compiler>>detectFeaturesOnLinux (in category 'feature detection') -----
detectFeaturesOnLinux
"Do throw-away compilations to read CTR_EL0 & ID_AA64ISAR0_EL1 and initialize ctrEl0 & idISAR0"
<notOption: #__APPLE__>
| startAddress getFeatureReg ctrEL0 idISAR0 |
<var: 'getFeatureReg' declareC: 'usqIntptr_t (*getFeatureReg)(void)'>
startAddress := cogit methodZoneBase.
cogit allocateOpcodes: 4 bytecodes: 0.
getFeatureReg := cogit cCoerceSimple: startAddress to: #'usqIntptr_t (*)(void)'.
"Return the value of CTR_EL0; that's the control register that defines the vital statistics of the processor's caches."
cogit
gen: Nop; "do something anodyne so it is easy to distinguish MRS_CTR_EL0 being an illegal instruction rather than the code zone not being executable."
gen: MRS_CTR_EL0 operand: ABIResultReg;
RetN: 0.
cogit outputInstructionsForGeneratedRuntimeAt: startAddress.
cogit resetMethodZoneBase: startAddress.
cogit ensureExecutableCodeZoneWithin:
[ctrEL0 := (self cCode: 'getFeatureReg()' inSmalltalk: [cogit simulateLeafCallOf: startAddress]).
+ "see e.g. CogARMv8Compiler class>>printCTR_EL0:, concretizeCacheControlOp1:CRm:Op2: &
+ http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.100403_0200_00_en/lau1443435580346.html
+ DminLine & IminLine are Log2 words; 16 words miniumum"
- "see e.g. CogARMv8Compiler class>>printCTR_EL0:"
self setDataCacheFlushRequired: (ctrEL0 noMask: 1 << 28).
self setDataCacheLineLength: 4 << (ctrEL0 >> 16 bitAnd: 15).
+ self dataCacheLineLength = 0 ifTrue:
+ [self setDataCacheLineLength: 4 << 4].
self setInstructionCacheFlushRequired: (ctrEL0 noMask: 1 << 29).
self setInstructionCacheLineLength: 4 << (ctrEL0 bitAnd: 15)].
+ self instructionCacheLineLength = 0 ifTrue:
+ [self setInstructionCacheLineLength: 4 << 4].
cogit zeroOpcodeIndexForNewOpcodes.
cogit
gen: Nop; "do something anodyne so it is easy to distinguish MRS_CTR_EL0 being an illegal instruction rather than the code zone not being executable."
gen: MRS_ID_AA64ISAR0_EL1 operand: ABIResultReg;
RetN: 0.
cogit outputInstructionsForGeneratedRuntimeAt: startAddress.
cogit resetMethodZoneBase: startAddress.
cogit ensureExecutableCodeZoneWithin:
[idISAR0 := (self cCode: 'getFeatureReg()' inSmalltalk: [cogit simulateLeafCallOf: startAddress]).
self setHasAtomicInstructions: (idISAR0 >> 20 bitAnd: 2r1111) = 2r10]!
Item was changed:
----- Method: CogARMv8Compiler>>flushICacheFrom:to: (in category 'inline cacheing') -----
flushICacheFrom: startAddress "<Integer>" to: endAddress "<Integer>"
"Flush the instruction cache from (startAddress to endAddress].
If there is a dual mapped code zone (the normal zone but marked with read/execute, and a
read/write zone codeToDataDelta bytes away) then also flush the data cache for the corresp-
onding range in the read/write zone and invalidate the data cache for the read/execute zone."
<inline: #always>
cogit ensureExecutableCodeZone.
+ self cCode:
+ [self cppIf: #__APPLE__
+ ifTrue: [self initialFlushICacheFrom: startAddress to: endAddress]
+ ifFalse:
+ [self cppIf: #__APPLE__
+ ifTrue: [cogit ceFlushDCache: startAddress _: endAddress].
+ cogit ceFlushICache: startAddress _: endAddress]]
- self cCode: [cogit ceFlushICache: startAddress _: endAddress]
inSmalltalk: [cogit simulateCeFlushICacheFrom: startAddress to: endAddress]!
Item was changed:
----- Method: CogARMv8Compiler>>generateDCacheFlush (in category 'inline cacheing') -----
generateDCacheFlush
"Use the DC instruction to implement ceFlushDCache(void *start, void *end); see flushDCacheFrom:to:.
If there is a dual mapped zone then clean data via DC_CVAU as address + codeToDataDelta,
then invalidate data at address via CIVAC."
"D4.4.7 About cache maintenance in AArch64 state D4-2478
Terminology for Clean, Invalidate, and Clean and Invalidate instructions D4-2479
...
- For instructions operating by VA, the following conceptual points are defined: D4-2480
Point of Unification (PoU)
The PoU for a PE is the point by which the instruction and data caches and the translation table walks of that
PE are guaranteed to see the same copy of a memory location. In many cases, the Point of Unification is the
point in a uniprocessor memory system by which the instruction and data caches and the translation table
walks have merged.
The PoU for an Inner Shareable shareability domain is the point by which the instruction and data caches
and the translation table walks of all the PEs in that Inner Shareable shareability domain are guaranteed to
see the same copy of a memory location. Defining this point permits self-modifying software to ensure future
instruction fetches are associated with the modified version of the software by using the standard correctness
policy of:
1. Clean data cache entry by address.
2. Invalidate instruction cache entry by address.
Example code for cache maintenance instructions D4-2490 - D4-2491"
+ | mask loop |
- | dataCacheMinLineLength mask loop |
self assert: cogit getCodeToDataDelta ~= 0.
- "See concretizeCacheControlOp1:CRm:Op2: &
- http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.100403_0200_00_en/lau1443435580346.html
- DminLine & IminLine are Log2 words; 16 words miniumum"
- (dataCacheMinLineLength := self dataCacheLineLength) = 0 ifTrue:
- [dataCacheMinLineLength := 4].
- dataCacheMinLineLength := 4 << dataCacheMinLineLength.
"Mask is large enough to encompass the method zone and has the correct minimum alignment."
+ mask := 1 << (cogit coInterpreter highBit: cogit methodZone zoneEnd) - self dataCacheLineLength.
- mask := 1 << (cogit coInterpreter highBit: cogit methodZone zoneEnd) - dataCacheMinLineLength.
"Since this is used from C code we must use only caller-saved registers.
C arg registers 2 & 3 are such a convenient pair of caller-saved registers."
cogit AndCq: mask R: CArg0Reg R: CArg2Reg. "CArg2Reg = aligned pointer to start of each data cache line"
cogit AddCq: cogit getCodeToDataDelta R: CArg2Reg R: CArg3Reg.
loop := cogit Label.
"see concretizeDataCacheControl"
cogit gen: DC operand: CArg3Reg operand: DC_CVAU. "clean (flush) address + codeToDataDelta"
cogit gen: DC operand: CArg2Reg operand: DC_CIVAC. "invalidate address"
cogit
+ AddCq: self dataCacheLineLength R: CArg2Reg;
+ AddCq: self dataCacheLineLength R: CArg3Reg;
- AddCq: dataCacheMinLineLength R: CArg2Reg;
- AddCq: dataCacheMinLineLength R: CArg3Reg;
CmpR: CArg1Reg R: CArg2Reg;
JumpBelowOrEqual: loop.
cogit RetN: 0!
Item was changed:
----- Method: CogARMv8Compiler>>generateICacheFlush (in category 'inline cacheing') -----
generateICacheFlush
"Use DC VAUC, DSB, IC IVAU, and ISB instructions to implement ceFlushICache(void *start, void *end); see flushICacheFrom:to:.
One might think that if there is a dual zone then data at address + codeToDataDelta must be cleaned,
but this isn't the case. All we need to do is clean data at address via DC VAUC and instructions via IC IVAU."
"B2.2.5 Concurrent modification and execution of instructions B2-112
...to avoid UNPREDICTABLE or CONSTRAINED UNPREDICTABLE behavior, instruction modifications must be explicitly synchronized before they are executed. The required synchronization is as follows:
1. No PE must be executing an instruction when another PE is modifying that instruction.
2. To ensure that the modified instructions are observable, a PE that is writing the instructions must issue the following sequence of instructions and operations:
; Coherency example for data and instruction accesses within the same Inner Shareable domain.
; enter this code with <Wt> containing a new 32-bit instruction, to be held in Cacheable space at a location pointed to by Xn.
STR Wt, [Xn]
DC CVAU, Xn ; Clean data cache by VA to point of unification (PoU)
DSB ISH ; Ensure visibility of the data cleaned from cache
IC IVAU, Xn ; Invalidate instruction cache by VA to PoU
DSB ISH
Note
- The DC CVAU operation is not required if the area of memory is either Non-cacheable or Write-Through Cacheable.
- If the contents of physical memory differ between the mappings, changing the mapping of VAs to PAs can cause
the instructions to be concurrently modified by one PE and executed by another PE. If the modifications affect
instructions other than those listed as being acceptable for modification, synchronization must be used to avoid
UNPREDICTABLE or CONSTRAINED UNPREDICTABLE behavior.
3. In a multiprocessor system, the IC IVAU is broadcast to all PEs within the Inner Shareable domain of the PE running this sequence.
However, when the modified instructions are observable, each PE that is executing the modified instructions must issue the following
instruction to ensure execution of the modified instructions:
ISB ; Synchronize fetched instruction stream"
"D4.4.7 About cache maintenance in AArch64 state D4-2478
Terminology for Clean, Invalidate, and Clean and Invalidate instructions D4-2479
...
- For instructions operating by VA, the following conceptual points are defined: D4-2480
Point of Unification (PoU)
The PoU for a PE is the point by which the instruction and data caches and the translation table walks of that
PE are guaranteed to see the same copy of a memory location. In many cases, the Point of Unification is the
point in a uniprocessor memory system by which the instruction and data caches and the translation table
walks have merged.
The PoU for an Inner Shareable shareability domain is the point by which the instruction and data caches
and the translation table walks of all the PEs in that Inner Shareable shareability domain are guaranteed to
see the same copy of a memory location. Defining this point permits self-modifying software to ensure future
instruction fetches are associated with the modified version of the software by using the standard correctness
policy of:
1. Clean data cache entry by address.
2. Invalidate instruction cache entry by address.
Example code for cache maintenance instructions D4-2490 - D4-2491"
+ | mask loop |
- | dataCacheMinLineLength instrCacheMinLineLength mask loop |
"See concretizeCacheControlOp1:CRm:Op2: &
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.100403_0200_00_en/lau1443435580346.html"
self dataCacheFlushRequired ifTrue: "CTR_EL0.IDC is zero; must clean data cache to point of unification."
["Since this is used from C code we must use only caller-saved registers.
C arg registers 2 & 3 are as such a convenient pair of caller-saved registers."
- dataCacheMinLineLength := self dataCacheLineLength.
"Mask is large enough to encompass the method zone and has the correct minimum alignment."
+ mask := 1 << (cogit coInterpreter highBit: cogit methodZone zoneEnd) - self dataCacheLineLength.
- mask := 1 << (cogit coInterpreter highBit: cogit methodZone zoneEnd) - dataCacheMinLineLength.
cogit AndCq: mask R: CArg0Reg R: CArg2Reg. "CArg2Reg = aligned pointer to start of each data cache line"
loop := cogit Label.
"see concretizeDataCacheControl"
cogit
gen: DC operand: CArg2Reg operand: DC_CVAU; "clean (flush) address"
+ AddCq: self dataCacheLineLength R: CArg2Reg;
- AddCq: dataCacheMinLineLength R: CArg2Reg;
CmpR: CArg1Reg R: CArg2Reg;
JumpBelowOrEqual: loop].
cogit gen: DSB operand: DSB_ISH operand: DSB_ALL. "Ensure visibility of the data cleaned from cache"
self instructionCacheFlushRequired ifTrue: "CTR_EL0.DIC is zero; must clean instruction cache to point of unification."
+ ["Mask is large enough to encompass the method zone and has the correct minimum alignment."
+ mask := 1 << (cogit coInterpreter highBit: cogit methodZone zoneEnd) - self instructionCacheLineLength.
- [instrCacheMinLineLength := self instructionCacheLineLength.
- "Mask is large enough to encompass the method zone and has the correct minimum alignment."
- mask := 1 << (cogit coInterpreter highBit: cogit methodZone zoneEnd) - instrCacheMinLineLength.
cogit AndCq: mask R: CArg0Reg R: CArg2Reg. "CArg2Reg = aligned pointer to start of each data cache line"
loop := cogit Label.
"see concretizeDataCacheControl"
cogit
gen: IC operand: CArg2Reg operand: IC_IVAU; "clean (flush) address"
+ AddCq: self instructionCacheLineLength R: CArg2Reg;
- AddCq: instrCacheMinLineLength R: CArg2Reg;
CmpR: CArg1Reg R: CArg2Reg;
JumpBelowOrEqual: loop.
cogit gen: DSB operand: DSB_ISH operand: DSB_ALL].
cogit
gen: ISB;
RetN: 0!
Item was changed:
----- Method: CogMethodZone>>ensureInYoungReferrers: (in category 'young referers') -----
ensureInYoungReferrers: cogMethod
<var: #cogMethod type: #'CogMethod *'>
cogit assertValidDualZoneReadAddress: cogMethod.
cogMethod cmRefersToYoung ifFalse:
[| writableMethod |
self assert: (self occurrencesInYoungReferrers: cogMethod) = 0.
+ self ensureWritableCodeZone.
(writableMethod := cogit writableMethodFor: cogMethod) cmRefersToYoung: true.
self addToYoungReferrers: writableMethod]!
Item was changed:
----- Method: CogMethodZone>>freeMethod: (in category 'compaction') -----
freeMethod: cogMethod
<var: #cogMethod type: #'CogMethod *'>
<inline: false>
| writableMethod |
self assert: cogMethod cmType ~= CMFree.
self assert: (cogit cogMethodDoesntLookKosher: cogMethod) = 0.
+ cogit ensureWritableCodeZone.
cogMethod cmType = CMMethod ifTrue:
["For non-Newspeak there should be a one-to-one mapping between bytecoded and
cog methods. For Newspeak not necessarily, but only for anonymous accessors."
"Only reset the original method's header if it is referring to this CogMethod."
(coInterpreter rawHeaderOf: cogMethod methodObject) asInteger = cogMethod asInteger
ifTrue:
[coInterpreter rawHeaderOf: cogMethod methodObject put: cogMethod methodHeader.
NewspeakVM ifTrue:
[(objectRepresentation canPinObjects and: [cogMethod nextMethodOrIRCs > self zoneEnd]) ifTrue:
[objectRepresentation freeIRCs: cogMethod nextMethodOrIRCs]]]
ifFalse:
[self cCode: [self assert: (cogit noAssertMethodClassAssociationOf: cogMethod methodObject) = objectMemory nilObject]
inSmalltalk: [self assert: ((cogit noAssertMethodClassAssociationOf: cogMethod methodObject) = objectMemory nilObject
or: [coInterpreter isKindOf: CurrentImageCoInterpreterFacade])].
NewspeakVM ifTrue:
[self removeFromUnpairedMethodList: cogMethod]].
cogit maybeFreeCountersOf: cogMethod].
cogMethod cmType = CMOpenPIC ifTrue:
[self removeFromOpenPICList: cogMethod].
writableMethod := cogit writableMethodFor: cogMethod.
writableMethod cmRefersToYoung: false.
writableMethod cmType: CMFree.
methodBytesFreedSinceLastCompaction := methodBytesFreedSinceLastCompaction
+ cogMethod blockSize!
Item was changed:
----- Method: CogMethodZone>>pruneYoungReferrers (in category 'young referers') -----
pruneYoungReferrers
| source dest next |
<api>
<var: #source type: #usqInt>
<var: #dest type: #usqInt>
<var: #next type: #usqInt>
<inline: false>
self assert: youngReferrers <= limitAddress.
youngReferrers = limitAddress ifTrue:
[^nil].
dest := limitAddress.
[next := dest - objectMemory wordSize.
next >= youngReferrers
and: [(coInterpreter cCoerceSimple: (objectMemory longAt: next) to: #'CogMethod *') cmRefersToYoung]] whileTrue:
[dest := next].
self assert: dest >= youngReferrers.
source := dest - objectMemory wordSize.
[source >= youngReferrers] whileTrue:
[(coInterpreter cCoerceSimple: (objectMemory longAt: source) to: #'CogMethod *') cmRefersToYoung ifTrue:
[self assert: source < (dest - objectMemory wordSize).
+ next ifNotNil: "convenient first-time flag"
+ [next := nil.
+ cogit ensureWritableCodeZone].
cogit codeLongAt: (dest := dest - objectMemory wordSize) put: (objectMemory longAt: source)].
source := source - objectMemory wordSize].
youngReferrers := dest.
self assert: self kosherYoungReferrers!
Item was changed:
----- Method: CogObjectRepresentationFor32BitSpur>>markAndTraceCacheTagLiteral:in:atpc: (in category 'garbage collection') -----
markAndTraceCacheTagLiteral: literal in: cogMethodOrNil atpc: address
"Mark and trace a literal in an inline cache preceding address in cogMethodOrNil.
Answer if code was modified."
<var: #cogMethodOrNil type: #'CogMethod *'>
<var: #address type: #usqInt>
| objOop |
(self couldBeObject: literal) ifFalse:
[^false].
self assert: (objectMemory addressCouldBeObj: literal).
(objectMemory isForwarded: literal) ifFalse:
[objectMemory markAndTrace: literal.
^false].
objOop := objectMemory followForwarded: literal.
+ self setCodeModified.
cogit backEnd rewriteInlineCacheTag: objOop at: address.
self markAndTraceUpdatedLiteral: objOop in: cogMethodOrNil.
^true!
Item was changed:
----- Method: CogObjectRepresentationForSpur>>markAndTraceLiteral:in:atpc: (in category 'garbage collection') -----
markAndTraceLiteral: literal in: cogMethodOrNil atpc: address
"Mark and trace a literal in a machine code instruction preceding address in cogMethodOrNil.
Answer if code was modified."
<var: #cogMethodOrNil type: #'CogMethod *'>
<var: #address type: #usqInt>
| objOop |
(self couldBeObject: literal) ifFalse:
[^false].
self assert: (objectMemory addressCouldBeObj: literal).
(objectMemory isForwarded: literal) ifFalse:
[objectMemory markAndTrace: literal.
^false].
+ cogit setCodeModified.
objOop := objectMemory followForwarded: literal.
cogit backEnd storeLiteral: objOop beforeFollowingAddress: address.
self markAndTraceUpdatedLiteral: objOop in: cogMethodOrNil.
^true!
Item was changed:
----- Method: CogVMSimulator>>ceTraceLinkedSend: (in category 'debug support') -----
ceTraceLinkedSend: theReceiver
(sendCount := sendCount + 1) \\ 500 = 0 ifTrue:
[self changed: #byteCountText].
cogit printOnTrace ifTrue:
[transcript print: byteCount; nextPut: $/; print: sendCount; space].
- cogit assertCStackWellAligned.
super ceTraceLinkedSend: theReceiver.
^#continue!
Item was changed:
----- Method: Cogit>>cogOpenPICSelector:numArgs: (in category 'in-line cacheing') -----
cogOpenPICSelector: selector numArgs: numArgs
"Create an Open PIC. Temporarily create a direct call of ceSendFromOpenPIC:.
Should become a probe of the first-level method lookup cache followed by a
call of ceSendFromOpenPIC: if the probe fails."
<returnTypeC: #'CogMethod *'>
| startAddress codeSize mapSize end |
coInterpreter compilationBreakpoint: selector isMNUCase: false.
startAddress := methodZone allocate: openPICSize.
startAddress = 0 ifTrue:
[^self cCoerceSimple: InsufficientCodeSpace to: #'CogMethod *'].
methodLabel
address: startAddress;
dependent: nil.
"stack allocate the various collections so that they
are effectively garbage collected on return."
self allocateOpcodes: 100 bytecodes: 0.
self compileOpenPIC: selector numArgs: numArgs.
self computeMaximumSizes.
methodLabel concretizeAt: startAddress.
codeSize := self generateInstructionsAt: startAddress + (self sizeof: CogMethod).
+ self ensureWritableCodeZone. "other clients first use generateMapAt: for sizing; this is an exception"
mapSize := self generateMapAt: startAddress + openPICSize - 1 start: startAddress + cmNoCheckEntryOffset.
self assert: entry address - startAddress = cmEntryOffset.
self assert: (methodZone roundUpLength: (self sizeof: CogMethod) + codeSize) + (methodZone roundUpLength: mapSize) <= openPICSize.
end := self outputInstructionsAt: startAddress + (self sizeof: CogMethod).
self
fillInOPICHeader: (self writableMethodFor: startAddress)
numArgs: numArgs
selector: selector.
^self cCoerceSimple: startAddress to: #'CogMethod *'!
Item was changed:
----- Method: Cogit>>compileSendTrace (in category 'debugging') -----
compileSendTrace
+ "2 is trace sends; 256 is traceLinkedSends"
+ <inline: #always>
+ ^traceFlags allMask: 256!
- "2 is trace sends; 256+2 is traceLinkedSends, so one can trace just unlinked sends using 2"
- <cmacro: '() ((traceFlags & 258) == 258)'>
- ^traceFlags allMask: 256 + 2!
Item was changed:
----- Method: Cogit>>followForwardedLiteralsIn: (in category 'garbage collection') -----
followForwardedLiteralsIn: cogMethod
<api>
<option: #SpurObjectMemory>
<var: #cogMethod type: #'CogMethod *'>
| writableCogMethod hasYoungObj hasYoungObjPtr |
self assert: (cogMethod cmType ~= CMMethod or: [(objectMemory isForwarded: cogMethod methodObject) not]).
writableCogMethod := self writableMethodFor: cogMethod.
+ self ensureWritableCodeZone.
hasYoungObj := objectMemory isYoung: cogMethod methodObject.
(objectMemory shouldRemapOop: cogMethod selector) ifTrue:
[writableCogMethod selector: (objectMemory remapObj: cogMethod selector).
(objectMemory isYoung: cogMethod selector) ifTrue:
[hasYoungObj := true]].
hasYoungObjPtr := (self addressOf: hasYoungObj put: [:val| hasYoungObj := val]) asInteger.
self mapFor: cogMethod
performUntil: #remapIfObjectRef:pc:hasYoung:
arg: hasYoungObjPtr.
hasYoungObj
ifTrue: [methodZone ensureInYoungReferrers: cogMethod]
ifFalse: [writableCogMethod cmRefersToYoung: false].
self ensureExecutableCodeZone!
Item was changed:
----- Method: Cogit>>followForwardedMethods (in category 'garbage collection') -----
followForwardedMethods
<api>
<option: #SpurObjectMemory>
- <var: #cogMethod type: #'CogMethod *'>
| cogMethod freedPIC |
+ self ensureWritableCodeZone.
- <var: #cogMethod type: #'CogMethod *'>
freedPIC := false.
cogMethod := self cCoerceSimple: methodZoneBase to: #'CogMethod *'.
[cogMethod < methodZone limitZony] whileTrue:
[cogMethod cmType = CMMethod ifTrue:
[(objectMemory isForwarded: cogMethod methodObject) ifTrue:
[cogMethod methodObject: (objectMemory followForwarded: cogMethod methodObject).
(objectMemory isYoungObject: cogMethod methodObject) ifTrue:
[methodZone ensureInYoungReferrers: cogMethod]]].
cogMethod cmType = CMClosedPIC ifTrue:
[(self followMethodReferencesInClosedPIC: cogMethod) ifTrue:
[freedPIC := true.
methodZone freeMethod: cogMethod]].
cogMethod := methodZone methodAfter: cogMethod].
freedPIC ifTrue:
+ [self unlinkSendsToFree].
+ self ensureExecutableCodeZone!
- [self unlinkSendsToFree]!
Item was changed:
----- Method: Cogit>>followMaybeObjRefInClosedPICAt: (in category 'garbage collection') -----
followMaybeObjRefInClosedPICAt: mcpc
"Follow a potential object reference from a closed PIC.
This may be a method reference or null.
Answer if the followed literal is young.
'mcpc' refers to the jump/branch instruction at the end of
each cpic case"
| object subject |
object := backEnd literalBeforeFollowingAddress: mcpc.
(objectRepresentation couldBeObject: object) ifFalse:
[^false].
(objectMemory isForwarded: object) ifFalse:
[^objectMemory isYoungObject: object].
subject := objectMemory followForwarded: object.
+ self setCodeModified.
backEnd storeLiteral: subject beforeFollowingAddress: mcpc.
- codeModified := true.
^objectMemory isYoungObject: subject!
Item was changed:
----- Method: Cogit>>followMovableLiteralsAndUpdateYoungReferrers (in category 'garbage collection') -----
followMovableLiteralsAndUpdateYoungReferrers
"To avoid runtime checks on literal variable and literal accesses in == and ~~,
we follow literals in methods having movable literals in the postBecome action.
To avoid scanning every method, we annotate cogMethods with the
cmHasMovableLiteral flag."
<option: #SpurObjectMemory>
<api>
<returnTypeC: #void>
| cogMethod |
<var: #cogMethod type: #'CogMethod *'>
self assert: methodZone kosherYoungReferrers.
"methodZone firstBogusYoungReferrer"
"methodZone occurrencesInYoungReferrers: methodZone firstBogusYoungReferrer"
codeModified := false.
cogMethod := self cCoerceSimple: methodZoneBase to: #'CogMethod *'.
[cogMethod < methodZone limitZony] whileTrue:
[cogMethod cmType ~= CMFree ifTrue:
[cogMethod cmHasMovableLiteral ifTrue:
[self followForwardedLiteralsIn: cogMethod]].
cogMethod := methodZone methodAfter: cogMethod]..
methodZone pruneYoungReferrers.
codeModified ifTrue: "After updating oops in inline caches we need to flush the icache."
+ [backEnd flushICacheFrom: codeBase asUnsignedInteger to: methodZone freeStart].
+ "And ensure code zone is executable. May have pruned young referrers..."
+ self ensureExecutableCodeZone!
- [backEnd flushICacheFrom: codeBase asUnsignedInteger to: methodZone freeStart]!
Item was changed:
----- Method: Cogit>>freeCogMethod: (in category 'jit - api') -----
freeCogMethod: cogMethod
<api>
<var: #cogMethod type: #'CogMethod *'>
- self ensureWritableCodeZone.
methodZone freeMethod: cogMethod.
self ensureExecutableCodeZone!
Item was changed:
----- Method: Cogit>>mapObjectReferencesInGeneratedRuntime (in category 'garbage collection') -----
mapObjectReferencesInGeneratedRuntime
"Update all references to objects in the generated runtime."
0 to: runtimeObjectRefIndex - 1 do:
[:i| | mcpc literal mappedLiteral |
mcpc := objectReferencesInRuntime at: i.
literal := literalsManager fetchLiteralAtAnnotatedAddress: mcpc using: backEnd.
mappedLiteral := objectRepresentation remapObject: literal.
mappedLiteral ~= literal ifTrue:
+ [self setCodeModified.
+ literalsManager storeLiteral: mappedLiteral atAnnotatedAddress: mcpc using: backEnd]]!
- [literalsManager storeLiteral: mappedLiteral atAnnotatedAddress: mcpc using: backEnd.
- codeModified := true]]!
Item was changed:
----- Method: Cogit>>markAndTraceOrFreeCogMethod:firstVisit: (in category 'garbage collection') -----
markAndTraceOrFreeCogMethod: cogMethod firstVisit: firstVisit
"Mark and trace objects in the argument and free if it is appropriate.
Answer if the method has been freed. firstVisit is a hint used to avoid
scanning methods we've already seen. False positives are fine.
For a CMMethod this
frees if the bytecode method isnt marked,
marks and traces object literals and selectors,
unlinks sends to targets that should be freed.
For a CMClosedPIC this
frees if it refers to anything that should be freed or isn't marked.
For a CMOpenPIC this
frees if the selector isn't marked."
<var: #cogMethod type: #'CogMethod *'>
<inline: false> "this recurses at most one level down"
cogMethod cmType = CMFree ifTrue:
[^true].
self assert: (self cogMethodDoesntLookKosher: cogMethod) = 0.
cogMethod cmType = CMMethod ifTrue:
[(objectMemory isMarked: cogMethod methodObject) ifFalse:
+ [self ensureWritableCodeZone.
+ methodZone freeMethod: cogMethod.
- [methodZone freeMethod: cogMethod.
^true].
firstVisit ifTrue:
[self markLiteralsAndUnlinkUnmarkedSendsIn: cogMethod].
^false].
cogMethod cmType = CMClosedPIC ifTrue:
[(self closedPICRefersToUnmarkedObject: cogMethod) ifFalse:
[^false].
+ self ensureWritableCodeZone.
methodZone freeMethod: cogMethod.
^true].
cogMethod cmType = CMOpenPIC ifTrue:
[(objectMemory isMarked: cogMethod selector) ifTrue:
[^false].
+ self ensureWritableCodeZone.
methodZone freeMethod: cogMethod.
^true].
self assert: (cogMethod cmType = CMMethod
or: [cogMethod cmType = CMClosedPIC
or: [cogMethod cmType = CMOpenPIC]]).
^false!
Item was changed:
----- Method: Cogit>>remapIfObjectRef:pc:hasYoung: (in category 'garbage collection') -----
remapIfObjectRef: annotation pc: mcpc hasYoung: hasYoungPtr
<var: #mcpc type: #'char *'>
<var: #targetMethod type: #'CogMethod *'>
<var: #nsSendCache type: #'NSSendCache *'>
annotation = IsObjectReference ifTrue:
[| literal mappedLiteral |
literal := literalsManager fetchLiteralAtAnnotatedAddress: mcpc asUnsignedInteger using: backEnd.
(objectRepresentation couldBeObject: literal) ifTrue:
[mappedLiteral := objectRepresentation remapObject: literal.
literal ~= mappedLiteral ifTrue:
+ [self setCodeModified.
+ literalsManager storeLiteral: mappedLiteral atAnnotatedAddress: mcpc asUnsignedInteger using: backEnd].
- [literalsManager storeLiteral: mappedLiteral atAnnotatedAddress: mcpc asUnsignedInteger using: backEnd.
- codeModified := true].
(hasYoungPtr ~= 0
and: [objectMemory isYoung: mappedLiteral]) ifTrue:
[(self cCoerceSimple: hasYoungPtr to: #'sqInt *') at: 0 put: true]]].
NewspeakVM ifTrue: [annotation = IsNSSendCall ifTrue:
[| nsSendCache oop mappedOop |
nsSendCache := self nsSendCacheFromReturnAddress: mcpc.
oop := nsSendCache selector.
mappedOop := objectRepresentation remapObject: oop.
oop ~= mappedOop ifTrue:
[nsSendCache selector: mappedOop.
(hasYoungPtr ~= 0 and: [objectMemory isYoung: mappedOop]) ifTrue:
[(self cCoerceSimple: hasYoungPtr to: #'sqInt *') at: 0 put: true]].
oop := nsSendCache enclosingObject.
oop ~= 0 ifTrue: [
mappedOop := objectRepresentation remapObject: oop.
oop ~= mappedOop ifTrue:
[nsSendCache enclosingObject: mappedOop.
(hasYoungPtr ~= 0 and: [objectMemory isYoung: mappedOop]) ifTrue:
[(self cCoerceSimple: hasYoungPtr to: #'sqInt *') at: 0 put: true]]].
^0 "keep scanning"]].
(self isPureSendAnnotation: annotation) ifTrue:
[self entryCacheTagAndCouldBeObjectAt: mcpc annotation: annotation into:
[:entryPoint :cacheTag :tagCouldBeObj | | mappedCacheTag |
(tagCouldBeObj
and: [objectRepresentation couldBeObject: cacheTag]) ifTrue:
[mappedCacheTag := objectRepresentation remapObject: cacheTag.
cacheTag ~= mappedCacheTag ifTrue:
+ [self setCodeModified.
+ backEnd rewriteInlineCacheTag: mappedCacheTag at: mcpc asUnsignedInteger].
- [backEnd rewriteInlineCacheTag: mappedCacheTag at: mcpc asUnsignedInteger.
- codeModified := true].
(hasYoungPtr ~= 0
and: [objectMemory isYoung: mappedCacheTag]) ifTrue:
[(self cCoerceSimple: hasYoungPtr to: #'sqInt *') at: 0 put: true]].
hasYoungPtr ~= 0 ifTrue:
["Since the unlinking routines may rewrite the cacheTag to the send's selector, and
since they don't have the cogMethod to hand and can't add it to youngReferrers,
the method must remain in youngReferrers if the targetMethod's selector is young."
entryPoint > methodZoneBase ifTrue: "It's a linked send."
[self targetMethodAndSendTableFor: entryPoint annotation: annotation into:
[:targetMethod :ignored|
(objectMemory isYoung: targetMethod selector) ifTrue:
[(self cCoerceSimple: hasYoungPtr to: #'sqInt *') at: 0 put: true]]]]]].
^0 "keep scanning"!
Item was changed:
----- Method: Cogit>>remapMaybeObjRefInClosedPICAt: (in category 'garbage collection') -----
remapMaybeObjRefInClosedPICAt: mcpc
"Remap a potential object reference from a closed PIC.
This may be an object reference, an inline cache tag or null.
Answer if the updated literal is young.
mcpc is the address of the next instruction following either
the load of the method literal or the compare of the class tag."
| object subject |
object := backEnd literalBeforeFollowingAddress: mcpc.
(objectRepresentation couldBeObject: object) ifFalse:
[^false].
subject := objectRepresentation remapOop: object.
object ~= subject ifTrue:
+ [self setCodeModified.
+ backEnd storeLiteral: subject beforeFollowingAddress: mcpc].
- [backEnd storeLiteral: subject beforeFollowingAddress: mcpc.
- codeModified := true].
^objectMemory isYoungObject: subject!
Item was added:
+ ----- Method: Cogit>>setCodeModified (in category 'memory access') -----
+ setCodeModified
+ "Three cases to avoid reading codeModified unless absolutely necessary. On some platforms run-time calls
+ may be required to enable execution and disable write-protect of the code zone. See the comment in ensureExecutableCodeZone."
+ <inline: #always>
+ self cppIf: #DUAL_MAPPED_CODE_ZONE
+ ifTrue: [codeModified := true]
+ ifFalse:
+ [backEnd needsCodeZoneExecuteWriteSwitch
+ ifTrue:
+ [codeModified ifFalse:
+ [codeModified := true.
+ backEnd makeCodeZoneWritable]]
+ ifFalse: [codeModified := true]]!
Item was changed:
----- Method: Cogit>>setInterpreter: (in category 'initialization') -----
setInterpreter: aCoInterpreter
"Initialization of the code generator in the simulator.
These objects already exist in the generated C VM
or are used only in the simulation."
<doNotGenerate>
coInterpreter := aCoInterpreter.
objectMemory := aCoInterpreter objectMemory.
methodZone := self class methodZoneClass new.
objectRepresentation := objectMemory objectRepresentationClass
forCogit: self methodZone: methodZone.
methodZone setInterpreter: aCoInterpreter
objectRepresentation: objectRepresentation
cogit: self.
generatorTable := self class generatorTable.
processor := ProcessorClass new.
simulatedAddresses := Dictionary new.
coInterpreter class clusteredVariableNames do:
[:cvn| self simulatedAddressFor: (cvn first = $C ifTrue: ['get', cvn] ifFalse: [cvn]) asSymbol].
simulatedTrampolines := Dictionary new.
simulatedVariableGetters := Dictionary new.
simulatedVariableSetters := Dictionary new.
traceStores := 0.
traceFlags := (InitializationOptions at: #linkedSendTrace ifAbsent: [false])
+ ifTrue: [257 "compileSendTrace + print"]
- ifTrue: [257]
ifFalse:
[(InitializationOptions at: #recordPrimTrace ifAbsent: [true])
ifTrue: [8] "record prim trace on by default (see Cogit class>>decareCVarsIn:)"
ifFalse: [0]].
singleStep := printRegisters := printInstructions := clickConfirm := false.
backEnd := CogCompilerClass for: self.
methodLabel := CogCompilerClass for: self.
(literalsManager := backEnd class literalsManagerClass new) cogit: self.
ordinarySendTrampolines := CArrayAccessor on: (Array new: NumSendTrampolines).
superSendTrampolines := CArrayAccessor on: (Array new: NumSendTrampolines).
BytecodeSetHasDirectedSuperSend ifTrue:
[directedSuperSendTrampolines := CArrayAccessor on: (Array new: NumSendTrampolines).
directedSuperBindingSendTrampolines := CArrayAccessor on: (Array new: NumSendTrampolines).
directedSendUsesBinding := false].
NewspeakVM ifTrue:
[selfSendTrampolines := CArrayAccessor on: (Array new: NumSendTrampolines).
dynamicSuperSendTrampolines := CArrayAccessor on: (Array new: NumSendTrampolines).
implicitReceiverSendTrampolines := CArrayAccessor on: (Array new: NumSendTrampolines).
outerSendTrampolines := CArrayAccessor on: (Array new: NumSendTrampolines)].
"debug metadata"
objectReferencesInRuntime := CArrayAccessor on: (Array new: NumObjRefsInRuntime).
runtimeObjectRefIndex := 0.
"debug metadata"
trampolineAddresses := CArrayAccessor on: (Array new: NumTrampolines * 2).
trampolineTableIndex := 0.
extA := numExtB := extB := 0.
compilationTrace ifNil: [compilationTrace := self class initializationOptions at: #compilationTrace ifAbsent: [0]].
debugOpcodeIndices := self class initializationOptions at: #debugOpcodeIndices ifAbsent: [Set new].
debugBytecodePointers := self class initializationOptions at: #debugBytecodePointers ifAbsent: [Set new].
self class initializationOptions at: #breakPC ifPresent: [:pc| breakPC := pc]!
Item was changed:
----- Method: Cogit>>shortcutTrampoline:to: (in category 'simulation only') -----
shortcutTrampoline: aProcessorSimulationTrap to: aBlock
<doNotGenerate>
"As a simulation performance hack for debugging trampolines such as ceTraceLinkedSend: allow the entire
trampoline to be implemented by this method instead of simulating all of the code for the trampoline."
+ | stsp |
+ stsp := processor registerAt: SPReg put: (processor registerAt: SPReg) - backEnd class wordSize.
+ "when the processor simulators are fixed to call the illegal address, and push the retpc on the stack, this will change to only push if there is a link reg."
+ objectMemory
+ longAt: stsp
+ put: (backEnd hasLinkRegister ifTrue: [processor lr] ifFalse: [aProcessorSimulationTrap nextpc]).
- backEnd hasLinkRegister ifTrue:
- [processor pushWord: processor lr in: coInterpreter memory].
- processor
- simulateLeafCallOf: aProcessorSimulationTrap address
- nextpc: aProcessorSimulationTrap nextpc
- memory: coInterpreter memory.
coInterpreter
+ stackPointer: (processor registerAt: SPReg);
- stackPointer: processor sp;
framePointer: processor fp.
- processor
- sp: coInterpreter getCStackPointer;
- fp: coInterpreter getCFramePointer.
aBlock value.
processor
+ registerAt: SPReg put: (processor registerAt: SPReg) + backEnd class wordSize;
+ pc: aProcessorSimulationTrap nextpc!
- sp: coInterpreter stackPointer;
- fp: coInterpreter framePointer;
- simulateLeafReturnIn: coInterpreter memory.
- backEnd hasLinkRegister ifTrue:
- [processor lr: (processor popWordIn: coInterpreter memory)]!
Item was changed:
----- Method: Cogit>>unlinkSendAt:targetMethod:sendTable: (in category 'in-line cacheing') -----
unlinkSendAt: mcpc targetMethod: targetMethod sendTable: sendTable
<inline: true>
| unlinkedRoutine |
unlinkedRoutine := sendTable at: (targetMethod cmNumArgs min: NumSendTrampolines - 1).
+ self setCodeModified.
backEnd
rewriteInlineCacheAt: mcpc asInteger
tag: (backEnd inlineCacheValueForSelector: targetMethod selector in: enumeratingCogMethod)
+ target: unlinkedRoutine!
- target: unlinkedRoutine.
- codeModified := true!
Item was changed:
----- Method: Cogit>>unlinkSendsOf:isMNUSelector: (in category 'jit - api') -----
unlinkSendsOf: selector isMNUSelector: isMNUSelector
<api>
"Unlink all sends in cog methods. Free all Closed PICs with the selector,
or with an MNU case if isMNUSelector. First check if any method actually
has the selector; if not there can't be any linked send to it. This routine
(including descendents) is performance critical. It contributes perhaps
30% of entire execution time in Compiler recompileAll."
| cogMethod mustScanAndUnlink |
<var: #cogMethod type: #'CogMethod *'>
methodZoneBase ifNil: [^self].
cogMethod := self cCoerceSimple: methodZoneBase to: #'CogMethod *'.
mustScanAndUnlink := false.
isMNUSelector
ifTrue:
[[cogMethod < methodZone limitZony] whileTrue:
[cogMethod cmType ~= CMFree ifTrue:
[cogMethod cpicHasMNUCase
ifTrue:
[self assert: cogMethod cmType = CMClosedPIC.
methodZone freeMethod: cogMethod.
mustScanAndUnlink := true]
ifFalse:
[cogMethod selector = selector ifTrue:
[mustScanAndUnlink := true.
cogMethod cmType = CMClosedPIC ifTrue:
[methodZone freeMethod: cogMethod]]]].
cogMethod := methodZone methodAfter: cogMethod]]
ifFalse:
[[cogMethod < methodZone limitZony] whileTrue:
[(cogMethod cmType ~= CMFree
and: [cogMethod selector = selector]) ifTrue:
[mustScanAndUnlink := true.
cogMethod cmType = CMClosedPIC ifTrue:
[methodZone freeMethod: cogMethod]].
cogMethod := methodZone methodAfter: cogMethod]].
mustScanAndUnlink ifFalse:
[^self].
codeModified := false.
cogMethod := self cCoerceSimple: methodZoneBase to: #'CogMethod *'.
[cogMethod < methodZone limitZony] whileTrue:
[cogMethod cmType = CMMethod ifTrue:
[self mapFor: cogMethod
performUntil: #unlinkIfFreeOrLinkedSend:pc:of:
arg: selector].
cogMethod := methodZone methodAfter: cogMethod].
codeModified ifTrue: "After possibly updating inline caches we need to flush the icache."
+ [backEnd flushICacheFrom: methodZoneBase asUnsignedInteger to: methodZone freeStart].
+ "And ensure code zone is executable. May merely have freed methods..."
+ self ensureExecutableCodeZone!
- [backEnd flushICacheFrom: methodZoneBase asUnsignedInteger to: methodZone freeStart]!
Item was changed:
----- Method: Cogit>>unlinkSendsTo:andFreeIf: (in category 'jit - api') -----
unlinkSendsTo: targetMethodObject andFreeIf: freeIfTrue
<api>
"Unlink all sends in cog methods to a particular target method.
If targetMethodObject isn't actually a method (perhaps being
used via invokeAsMethod) then there's nothing to do."
| cogMethod targetMethod freedPIC |
<var: #cogMethod type: #'CogMethod *'>
<var: #targetMethod type: #'CogMethod *'>
((objectMemory isOopCompiledMethod: targetMethodObject)
and: [coInterpreter methodHasCogMethod: targetMethodObject]) ifFalse:
[^self].
targetMethod := coInterpreter cogMethodOf: targetMethodObject.
methodZoneBase ifNil: [^self].
+ self ensureWritableCodeZone.
codeModified := freedPIC := false.
cogMethod := self cCoerceSimple: methodZoneBase to: #'CogMethod *'.
[cogMethod < methodZone limitZony] whileTrue:
[cogMethod cmType = CMMethod
ifTrue:
[self mapFor: cogMethod
performUntil: #unlinkIfLinkedSend:pc:to:
arg: targetMethod asInteger]
ifFalse:
[(cogMethod cmType = CMClosedPIC
and: [self cPIC: cogMethod HasTarget: targetMethod]) ifTrue:
[methodZone freeMethod: cogMethod.
freedPIC := true]].
cogMethod := methodZone methodAfter: cogMethod].
freeIfTrue ifTrue: [self freeMethod: targetMethod].
freedPIC
ifTrue: [self unlinkSendsToFree]
ifFalse:
[codeModified ifTrue: "After possibly updating inline caches we need to flush the icache."
+ [backEnd flushICacheFrom: methodZoneBase asUnsignedInteger to: methodZone freeStart]].
+ self ensureExecutableCodeZone!
- [backEnd flushICacheFrom: methodZoneBase asUnsignedInteger to: methodZone freeStart]]!
More information about the Vm-dev
mailing list