[Vm-dev] VM Maker: VMMaker.oscog-eem.1881.mcz

commits at source.squeak.org commits at source.squeak.org
Mon Jun 6 19:57:16 UTC 2016


Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.1881.mcz

==================== Summary ====================

Name: VMMaker.oscog-eem.1881
Author: eem
Time: 6 June 2016, 12:55:12.14917 pm
UUID: c51b1764-ddf8-49fc-92bd-d587b3d65101
Ancestors: VMMaker.oscog-eem.1880

On Spur, handle external primitive failures answering PrimErrNoMemory by doing a scavenge and retrying and then on subsequent PrimErrNoMemory failure a full GC and then retrying.  Rename maybeRetryFailureDueToForwarding to maybeRetryPrimitiveOnFailure.

In the Cogit, change the retry mechanism so that the primitive call is made in the run-time, instead of the run-time answering whether the primitive should be retried and returning to machine code that then retries.

In the simulator, go through the contortions to map[ back whatever primitiveFunctionPointer is to that which dispatchFunctionPointer: accepts.

Discussion:
First, this mechanism applies only to external (plugin) primitives.  Spur's core primtives are written in the expectation that image-level primitive failure code will deal with the failure and retry.  This corresponds to the VisualWorks style and allows the image to implement GC policy (e.g. to favour reclamation over growth).  So with the current Spur code, heavily influenced by the author's experience with VisualWorks, it would be inapproprate to retry corte primitives on being out-of-memory.  The pathology is that attempts at very large allocations which will inevitably fail will cause a GC in the VM, very possibly followed by a GC from the image level primitive failure code.  Whereas leaving it to the image, the image code may be intellkigent enough to spot an allocation too large to ever succeed and avoid the GC altogether.

I'm not sure if I like this auto-retry or not.  Perhaps it is a good idea and the image-level primitive failure code should be changed; there would certainly be fewer image-level methods.  But that means fewer hooks for the image to implement GC policy.  retryPrimitiveOnFailure which manages the retry isn't exactly simple (mapping back the primitiveFunctionPointer to something dispatchFunctionPointer: can manage isn't simple either, but that's simulation only, and so its complications carry much less weight).

=============== Diff against VMMaker.oscog-eem.1880 ===============

Item was changed:
  ----- Method: CoInterpreter>>ceActivateFailingPrimitiveMethod: (in category 'enilopmarts') -----
  ceActivateFailingPrimitiveMethod: aPrimitiveMethod
  	"An external call or FFI primitive has failed.  Build the frame and
  	 activate as appropriate.  Enter either the interpreter or machine
  	 code depending on whether aPrimitiveMethod has been or is still
  	 cogged.  Note that we could always interpret but want the efficiency
  	 of executing machine code if it is available."
  	<api>
  	| methodHeader result |
  	self assert: primFailCode ~= 0.
  	self assert: newMethod = aPrimitiveMethod.
+ 	"If we're on Spur, retry the primitive, if appropriate,
+ 	 returning if successful after retry."
+ 	objectMemory hasSpurMemoryManagerAPI ifTrue:
+ 		[self retryPrimitiveOnFailure.
- 	"If we're on Spur, check for forwarders and retry,
- 	 returning if successful the second time around."
- 	(objectMemory hasSpurMemoryManagerAPI
- 	 and: [self checkForAndFollowForwardedPrimitiveState]) ifTrue:
- 		[self initPrimCall.
- 		 self cCode: [self dispatchFunctionPointer: primitiveFunctionPointer]
- 			inSmalltalk:
- 				[| evaluable |
- 				 evaluable := primitiveFunctionPointer isInteger
- 								ifTrue: [cogit simulatedTrampolines at: primitiveFunctionPointer]
- 								ifFalse: [primitiveFunctionPointer].
- 				 evaluable isMessageSend
- 					ifTrue: [self assert: evaluable receiver == self]
- 					ifFalse: [self assert: evaluable isBlock].
- 				 evaluable value].
  		 self successful ifTrue:
  			[result := self stackTop.
  			 self stackTopPut: instructionPointer.
  			 self push: result.
  			 cogit ceEnterCogCodePopReceiverReg]].
  	methodHeader := self rawHeaderOf: aPrimitiveMethod.
  	(self isCogMethodReference: methodHeader)
  		ifTrue: [self activateCoggedNewMethod: false]
  		ifFalse: [self activateNewMethod]!

Item was added:
+ ----- Method: CoInterpreter>>ceCheckAndMaybeRetryPrimitive: (in category 'primitive support') -----
+ ceCheckAndMaybeRetryPrimitive: primIndex
+ 	"Log failure and then retry if there's an accessorDepth or failure due to no memory."
+ 	<option: #SpurObjectMemory>
+ 	| retried |
+ 	cogit recordPrimTrace ifTrue:
+ 		[self fastLogPrim: TracePrimitiveFailure].
+ 	retried := self retryPrimitiveOnFailure.
+ 	(retried and: [cogit recordPrimTrace]) ifTrue:
+ 		[self fastLogPrim: TracePrimitiveRetry]!

Item was removed:
- ----- Method: CoInterpreter>>ceCheckForAndFollowForwardedPrimitiveState (in category 'cog jit support') -----
- ceCheckForAndFollowForwardedPrimitiveState
- 	"In Spur a primitive may fail due to encountering a forwarder.
- 	 On failure check the accessorDepth for the primitive and
- 	 if non-negative scan the args to the depth, following any
- 	 forwarders.  Answer if any are found so the prim can be retried."
- 	<api>
- 	<option: #SpurObjectMemory>
- 	^self cCode: [self checkForAndFollowForwardedPrimitiveState]
- 		inSmalltalk: [(self checkForAndFollowForwardedPrimitiveState)
- 						ifTrue: [1]
- 						ifFalse: [0]]!

Item was added:
+ ----- Method: CoInterpreter>>primNumberExternalCall (in category 'compiled methods') -----
+ primNumberExternalCall
+ 	"Answer if the method is an external primtiive call (prim 117)."
+ 	<api>
+ 	<cmacro>
+ 	^PrimNumberExternalCall!

Item was removed:
- ----- Method: CogObjectRepresentation>>maybeCompileRetry:onPrimitiveFail: (in category 'primitive generators') -----
- maybeCompileRetry: retryInst onPrimitiveFail: primIndex
- 	<var: #retryInst type: #'AbstractInstruction *'>
- 	"Object representations with lazy forwarding will want to check for
- 	 forwarding pointers on primitive failure and retry the primitive if found.
- 	 By default do nothing."!

Item was added:
+ ----- Method: CogObjectRepresentation>>maybeCompileRetryOnPrimitiveFail: (in category 'primitive generators') -----
+ maybeCompileRetryOnPrimitiveFail: primIndex
+ 	"Object representations with lazy forwarding will want to check for
+ 	 forwarding pointers on primitive failure and retry the primitive if found.
+ 	 By default do nothing."
+ 	<inline: true>!

Item was removed:
- ----- Method: CogObjectRepresentationForSpur>>maybeCompileRetry:onPrimitiveFail: (in category 'primitive generators') -----
- maybeCompileRetry: retryInst onPrimitiveFail: primIndex
- 	<var: #retryInst type: #'AbstractInstruction *'>
- 	"If primIndex has an accessorDepth, check for primitive failure and call
- 	 ceCheckForAndFollowForwardedPrimitiveState if so  If ceCheck.... answers
- 	 true, retry the primitive."
- 	| jmp |
- 	<var: #jmp type: #'AbstractInstruction *'>
- 	(coInterpreter accessorDepthForPrimitiveIndex: primIndex) < 0 ifTrue:
- 		[^0].
- 	cogit MoveAw: coInterpreter primFailCodeAddress R: TempReg.
- 	cogit CmpCq: 0 R: TempReg.
- 	jmp := cogit JumpZero: 0.
- 	cogit
- 		compileCallFor: #ceCheckForAndFollowForwardedPrimitiveState
- 		numArgs: 0
- 		arg: nil
- 		arg: nil
- 		arg: nil
- 		arg: nil
- 		resultReg: TempReg
- 		regsToSave: cogit emptyRegisterMask.
- 	cogit CmpCq: 0 R: TempReg.
- 	cogit JumpNonZero: retryInst.
- 	jmp jmpTarget: cogit Label.
- 	^0!

Item was added:
+ ----- Method: CogObjectRepresentationForSpur>>maybeCompileRetryOnPrimitiveFail: (in category 'primitive generators') -----
+ maybeCompileRetryOnPrimitiveFail: primIndex
+ 	<var: #retryInst type: #'AbstractInstruction *'>
+ 	"If primIndex has an accessorDepth and fails, or it is external and fails with PrimErrNoMemory,
+ 	 call ceCheckAndMaybeRetryPrimitive if so  If ceCheck.... answers true, retry the primitive."
+ 	| jmp |
+ 	<var: #jmp type: #'AbstractInstruction *'>
+ 	(coInterpreter accessorDepthForPrimitiveIndex: primIndex) >= 0
+ 		ifTrue:
+ 			[jmp := cogit
+ 				MoveAw: coInterpreter primFailCodeAddress R: TempReg;
+ 				CmpCq: 0 R: TempReg;
+ 				JumpZero: 0]
+ 		ifFalse:
+ 			[coInterpreter primNumberExternalCall ~= primIndex ifTrue:
+ 				[^0].
+ 			 jmp := cogit
+ 				MoveAw: coInterpreter primFailCodeAddress R: TempReg;
+ 				CmpCq: PrimErrNoMemory R: TempReg;
+ 				JumpNonZero: 0].
+ 	cogit
+ 		compileCallFor: #ceCheckAndMaybeRetryPrimitive:
+ 		numArgs: 1
+ 		arg: (cogit trampolineArgConstant: primIndex)
+ 		arg: nil
+ 		arg: nil
+ 		arg: nil
+ 		resultReg: TempReg
+ 		regsToSave: cogit emptyRegisterMask.
+ 	jmp jmpTarget: cogit Label.
+ 	^0!

Item was added:
+ ----- Method: CogVMSimulator>>mappedPluginEntries (in category 'plugin support') -----
+ mappedPluginEntries
+ 	^mappedPluginEntries!

Item was added:
+ ----- Method: CogVMSimulator>>maybeMapPrimitiveFunctionPointerBackToSomethingEvaluable (in category 'primitive support') -----
+ maybeMapPrimitiveFunctionPointerBackToSomethingEvaluable
+ 	"In the real VM primitiveFunctionPointer is either an index (for quick primitives)
+ 	 or a proper function pointer to a primitive.  In the simulator it may be a small
+ 	 index (corresponding to a quick primitive index), a symbol (corresponding to
+ 	 a function pointer) or an index into the externalPrimitiveTable, or an invalid
+ 	 address that references an evaluable in the simulatedTrampolines dictionary
+ 	 of the Cogit.  The simulator expects dispatchFunctionPointer to be called with
+ 	 primitiveFunctionPointer being a symbol only for internal primitives.  External
+ 	 primitives must have their funciton pointer mapped back to an index.  This
+ 	 method does the mapping back from fake addresses."
+ 	<doNotGenerate>
+ 	(primitiveFunctionPointer isInteger
+ 	 and: [self isExternalPrimitiveCall: newMethod]) ifTrue: "External prims must be evaluated by the right plugin..."
+ 		[(cogit simulatedTrampolines at: primitiveFunctionPointer ifAbsent: nil) ifNotNil:
+ 			[:evaluable| | pfp index externalIndex |
+ 						"primitiveFunctionPointer := pfp"
+ 						"(1 to: self mappedPluginEntries size) select: [:index| (self mappedPluginEntries at: index) third == evaluable]"
+ 			 pfp := primitiveFunctionPointer.
+ 			 index := self mappedPluginEntries findFirst: [:entry| entry third == evaluable].
+ 			 self assert: index ~= 0.
+ 			 externalIndex := 1000 + (externalPrimitiveTable object
+ 										indexOf: index
+ 										ifAbsent: [self error: 'entry not found']).
+ 			 self assert: ((self pluginEntryFor: externalIndex) notNil
+ 						   and: [(self pluginEntryFor: externalIndex) third == evaluable]).
+ 			 primitiveFunctionPointer := externalIndex.
+ 			 ^self]].
+ 	^super maybeMapPrimitiveFunctionPointerBackToSomethingEvaluable!

Item was added:
+ ----- Method: CurrentImageCoInterpreterFacadeForSpurObjectRepresentation>>ceCheckAndMaybeRetryPrimitive: (in category 'accessing') -----
+ ceCheckAndMaybeRetryPrimitive: primIndex
+ 	^coInterpreter ceCheckAndMaybeRetryPrimitive: primIndex!

Item was removed:
- ----- Method: CurrentImageCoInterpreterFacadeForSpurObjectRepresentation>>ceCheckForAndFollowForwardedPrimitiveState (in category 'accessing') -----
- ceCheckForAndFollowForwardedPrimitiveState
- 	^coInterpreter ceCheckForAndFollowForwardedPrimitiveState!

Item was changed:
  ----- Method: SimpleStackBasedCogit>>compileInterpreterPrimitive: (in category 'primitive generators') -----
  compileInterpreterPrimitive: primitiveRoutine
  	"Compile a call to an interpreter primitive.  Call the C routine with the
  	 usual stack-switching dance, test the primFailCode and then either
  	 return on success or continue to the method body."
  	<var: #primitiveRoutine declareC: 'void (*primitiveRoutine)(void)'>
+ 	| flags jmp jmpSamplePrim continuePostSamplePrim jmpSampleNonPrim continuePostSampleNonPrim |
- 	| flags jmp jmpSamplePrim retry continuePostSamplePrim jmpSampleNonPrim continuePostSampleNonPrim |
  	<var: #jmp type: #'AbstractInstruction *'>
- 	<var: #retry type: #'AbstractInstruction *'>
  	<var: #jmpSamplePrim type: #'AbstractInstruction *'>
- 	<var: #continuePostSamplePrim type: #'AbstractInstruction *'>
  	<var: #jmpSampleNonPrim type: #'AbstractInstruction *'>
+ 	<var: #continuePostSamplePrim type: #'AbstractInstruction *'>
  	<var: #continuePostSampleNonPrim type: #'AbstractInstruction *'>
  
  	"Save processor fp, sp and return pc in the interpreter's frame stack and instruction pointers"
  	self genExternalizePointersForPrimitiveCall.
  	"Switch to the C stack."
  	self genLoadCStackPointersForPrimCall.
  
  	flags := coInterpreter primitivePropertyFlags: primitiveIndex.
  	(flags anyMask: PrimCallDoNotJIT) ifTrue:
  		[^ShouldNotJIT].
  
  	(flags anyMask: PrimCallCollectsProfileSamples) ifTrue:
  		["Test nextProfileTick for being non-zero and call checkProfileTick if so"
  		objectMemory wordSize = 4
  			ifTrue:
  				[self MoveAw: coInterpreter nextProfileTickAddress R: TempReg.
  				 self MoveAw: coInterpreter nextProfileTickAddress + objectMemory wordSize R: ClassReg.
  				 self OrR: TempReg R: ClassReg]
  			ifFalse:
  				[self MoveAw: coInterpreter nextProfileTickAddress R: TempReg.
  				 self CmpCq: 0 R: TempReg].
  		"If set, jump to record sample call."
  		jmpSampleNonPrim := self JumpNonZero: 0.
  		continuePostSampleNonPrim := self Label].
  
  	"Old full prim trace is in VMMaker-eem.550 and prior"
  	self recordPrimTrace ifTrue:
  		[self genFastPrimTraceUsing: ClassReg and: SendNumArgsReg].
  
  	"Clear the primFailCode and set argumentCount"
+ 	self MoveCq: 0 R: TempReg.
- 	retry := self MoveCq: 0 R: TempReg.
  	self MoveR: TempReg Aw: coInterpreter primFailCodeAddress.
  	methodOrBlockNumArgs ~= 0 ifTrue:
  		[self MoveCq: methodOrBlockNumArgs R: TempReg].
  	self MoveR: TempReg Aw: coInterpreter argumentCountAddress.
  
  	"If required, set primitiveFunctionPointer and newMethod"
  	(flags anyMask: PrimCallNeedsPrimitiveFunction) ifTrue:
  		[self MoveCw: primitiveRoutine asInteger R: TempReg.
  		 primSetFunctionLabel :=
  		 self MoveR: TempReg Aw: coInterpreter primitiveFunctionPointerAddress].
  	(flags anyMask: PrimCallNeedsNewMethod+PrimCallMayCallBack) ifTrue:
  		["The ceActivateFailingPrimitiveMethod: machinery can't handle framelessness."
  		 (flags anyMask: PrimCallMayCallBack) ifTrue:
  			[needsFrame := true].
  		 methodLabel addDependent:
  			(self annotateAbsolutePCRef:
  				(self MoveCw: methodLabel asInteger R: ClassReg)).
  		 self MoveMw: (self offset: CogMethod of: #methodObject) r: ClassReg R: TempReg.
  		 self MoveR: TempReg Aw: coInterpreter newMethodAddress].
  
  	"Invoke the primitive"
  	self PrefetchAw: coInterpreter primFailCodeAddress.
  	(flags anyMask: PrimCallMayCallBack)
  		ifTrue: "Sideways call the C primitive routine so that we return through cePrimReturnEnterCogCode."
  			["On Spur ceActivateFailingPrimitiveMethod: would like to retry if forwarders
  			  are found. So insist on PrimCallNeedsPrimitiveFunction being set too."
  			 self assert: (flags anyMask: PrimCallNeedsPrimitiveFunction).
  			 backEnd genSubstituteReturnAddress:
  				((flags anyMask: PrimCallCollectsProfileSamples)
  					ifTrue: [cePrimReturnEnterCogCodeProfiling]
  					ifFalse: [cePrimReturnEnterCogCode]).
  			 primInvokeInstruction := self JumpFullRT: primitiveRoutine asInteger.
  			 jmp := jmpSamplePrim := continuePostSamplePrim := nil]
  		ifFalse:
  			["Call the C primitive routine."
  			primInvokeInstruction := self CallFullRT: primitiveRoutine asInteger.
  			(flags anyMask: PrimCallCollectsProfileSamples) ifTrue:
  				[self assert: (flags anyMask: PrimCallNeedsNewMethod).
  				"Test nextProfileTick for being non-zero and call checkProfileTick if so"
  				objectMemory wordSize = 4
  					ifTrue:
  						[self MoveAw: coInterpreter nextProfileTickAddress R: TempReg.
  						 self MoveAw: coInterpreter nextProfileTickAddress + objectMemory wordSize R: ClassReg.
  						 self OrR: TempReg R: ClassReg]
  					ifFalse:
  						[self MoveAw: coInterpreter nextProfileTickAddress R: TempReg.
  						 self CmpCq: 0 R: TempReg].
  				"If set, jump to record sample call."
  				jmpSamplePrim := self JumpNonZero: 0.
  				continuePostSamplePrim := self Label].
+ 			objectRepresentation maybeCompileRetryOnPrimitiveFail: primitiveIndex.
- 			objectRepresentation maybeCompileRetry: retry onPrimitiveFail: primitiveIndex.
  			self maybeCompileAllocFillerCheck.
  			"Switch back to the Smalltalk stack.  Stack better be in either of these two states:
  				success:	stackPointer ->	result (was receiver)
  											arg1
  											...
  											argN
  											return pc
  				failure:						receiver
  											arg1
  											...
  							stackPointer ->	argN
  											return pc
  			In either case we can push the instructionPointer or load it into the LinkRegister to reestablish the return pc"
  			self MoveAw: coInterpreter instructionPointerAddress
  				R: (backEnd hasLinkRegister ifTrue: [LinkReg] ifFalse: [ClassReg]).
  			backEnd genLoadStackPointers.
  			"Test primitive failure"
  			self MoveAw: coInterpreter primFailCodeAddress R: TempReg.
  			backEnd hasLinkRegister ifFalse: [self PushR: ClassReg]. "Restore return pc on CISCs"
  			self flag: 'ask concrete code gen if move sets condition codes?'.
  			self CmpCq: 0 R: TempReg.
  			jmp := self JumpNonZero: 0.
  			"Fetch result from stack"
  			self MoveMw: (backEnd hasLinkRegister ifTrue: [0] ifFalse: [objectMemory wordSize])
  				r: SPReg
  				R: ReceiverResultReg.
  			self RetN: objectMemory wordSize].	"return to caller, popping receiver"
  
  	(flags anyMask: PrimCallCollectsProfileSamples) ifTrue:
  		["The sample is collected by cePrimReturnEnterCogCode for external calls"
  		jmpSamplePrim ifNotNil:
  			["Call ceCheckProfileTick: to record sample and then continue."
  			jmpSamplePrim jmpTarget: self Label.
  			self assert: (flags anyMask: PrimCallNeedsNewMethod).
  			self CallFullRT: (self cCode: [#ceCheckProfileTick asUnsignedLong]
  							   inSmalltalk: [self simulatedTrampolineFor: #ceCheckProfileTick]).
  			"reenter the post-primitive call flow"
  			self Jump: continuePostSamplePrim].
  		"Null newMethod and call ceCheckProfileTick: to record sample and then continue.
  		 ceCheckProfileTick will map null/0 to coInterpreter nilObject"
  		jmpSampleNonPrim jmpTarget: self Label.
  		self MoveCq: 0 R: TempReg.
  		self MoveR: TempReg Aw: coInterpreter newMethodAddress.
  		self CallFullRT: (self cCode: [#ceCheckProfileTick asUnsignedLong]
  						   inSmalltalk: [self simulatedTrampolineFor: #ceCheckProfileTick]).
  		"reenter the post-primitive call flow"
  		self Jump: continuePostSampleNonPrim].
  
  	jmp ifNotNil:
  		["Jump to restore of receiver reg and proceed to frame build for failure."
  		 jmp jmpTarget: self Label.
  		 "Restore receiver reg from stack.  If on RISCs ret pc is in LinkReg, if on CISCs ret pc is on stack."
  		 self MoveMw: objectMemory wordSize * (methodOrBlockNumArgs + (backEnd hasLinkRegister ifTrue: [0] ifFalse: [1]))
  			r: SPReg
  			R: ReceiverResultReg].
  	^0!

Item was changed:
  CogClass subclass: #SpurMemoryManager
(excessive size, no diff calculated)

Item was added:
+ ----- Method: StackInterpreter>>maybeMapPrimitiveFunctionPointerBackToSomethingEvaluable (in category 'primitive support') -----
+ maybeMapPrimitiveFunctionPointerBackToSomethingEvaluable
+ 	"In the real VM primitiveFunctionPointer is either an index (for quick primitives)
+ 	 or a proper function pointer to a primitive.  In the simulator it may be a small
+ 	 index (corresponding to a quick primitive index), a symbol (corresponding to
+ 	 a function pointer) or an index into the externalPrimitiveTable, or an invalid
+ 	 address that references an evaluable in the simulatedTrampolines dictionary
+ 	 of the Cogit.  The simulator expects dispatchFunctionPointer to be called with
+ 	 primitiveFunctionPointer being a symbol only for internal primitives.  External
+ 	 primitives must have their funciton pointer mapped back to an index.  This
+ 	 method does the reverse mapping."
+ 	<doNotGenerate>
+ 	(self isExternalPrimitiveCall: newMethod) ifTrue: "External prims must be evaluated by the right plugin..."
+ 		[| pfp index externalIndex |
+ 		 pfp := primitiveFunctionPointer.
+ 		 index := self mappedPluginEntries findFirst: [:entry| entry second == primitiveFunctionPointer].
+ 		 self assert: index ~= 0.
+ 		 externalIndex := 1000 + (externalPrimitiveTable object
+ 												indexOf: index
+ 												ifAbsent: [self error: 'entry not found']).
+ 		 self assert: ((self pluginEntryFor: externalIndex) notNil
+ 					   and: [(self pluginEntryFor: externalIndex) second == primitiveFunctionPointer]).
+ 		 primitiveFunctionPointer := externalIndex]!

Item was removed:
- ----- Method: StackInterpreter>>maybeRetryFailureDueToForwarding (in category 'primitive support') -----
- maybeRetryFailureDueToForwarding
- 	"In Spur a primitive may fail due to encountering a forwarder. On failure, check
- 	 the accessorDepth for the primitive and if non-negative scan the args to the
- 	 depth, following any forwarders.  Retry the primitive if any are found."
- 	<inline: true>
- 	(objectMemory hasSpurMemoryManagerAPI
- 	 and: [self failed
- 	 and: [self checkForAndFollowForwardedPrimitiveState]]) ifTrue:
- 		[self initPrimCall.
- 		 self dispatchFunctionPointer: primitiveFunctionPointer]!

Item was added:
+ ----- Method: StackInterpreter>>maybeRetryPrimitiveOnFailure (in category 'primitive support') -----
+ maybeRetryPrimitiveOnFailure
+ 	"In Spur two cases of pirmitive failure are handled specially.  A primitive may fail due to validation
+ 	 encountering a forwarder. On failure, check the accessorDepth for the primitive and if non-negative
+ 	 scan the args to the depth, following any forwarders.  Retry the primitive if any are found.  Hence
+ 	 lazily and transparently following forwarders on primtiive failue.  Additionally a prmitive might fail
+ 	 due to an allocation failing.  Retry if primitives have failed with PrimErrNoMemory after running
+ 	 first the scavenger and then on a subsequent failure, the global mark-sweep collector.  Hence lazily
+ 	 and transparently GC on memory exhaustion."
+ 	<inline: true>
+ 	(objectMemory hasSpurMemoryManagerAPI and: [self failed]) ifTrue:
+ 		[self retryPrimitiveOnFailure]!

Item was added:
+ ----- Method: StackInterpreter>>retryPrimitiveOnFailure (in category 'primitive support') -----
+ retryPrimitiveOnFailure
+ 	"In Spur two cases of pirmitive failure are handled specially.  A primitive may fail due to validation
+ 	 encountering a forwarder. On failure, check the accessorDepth for the primitive and if non-negative
+ 	 scan the args to the depth, following any forwarders.  Retry the primitive if any are found.  Hence
+ 	 lazily and transparently following forwarders on primtiive failure.  Additionally a prmitive might fail
+ 	 due to an allocation failing.  Retry if external primitives have failed with PrimErrNoMemory after running
+ 	 first the scavenger and then on a subsequent failure, the global mark-sweep collector.  Hence lazily
+ 	 and transparently GC on memory exhaustion."
+ 	<inline: true>
+ 	| gcDone followDone canRetry retry retried |
+ 	gcDone := 0.
+ 	followDone := canRetry := retried := false.
+ 	[retry := false.
+ 	 primFailCode = PrimErrNoMemory
+ 		ifTrue:
+ 			[(gcDone := gcDone + 1) = 1 ifTrue:
+ 				[canRetry := self isExternalPrimitiveCall: newMethod].
+ 			 canRetry ifTrue:
+ 				 [gcDone = 1 ifTrue:
+ 					[objectMemory scavengingGC].
+ 				 gcDone = 2 ifTrue:
+ 					[objectMemory fullGC].
+ 				 retry := gcDone <= 2]]
+ 		 ifFalse:
+ 			[followDone ifFalse:
+ 				[followDone := true.
+ 				 retry := self checkForAndFollowForwardedPrimitiveState]].
+ 	 retry] whileTrue:
+ 		[self assert: primFailCode ~= 0.
+ 		 retried := true.
+ 		 self initPrimCall.
+ 		 self cCode: [] inSmalltalk:
+ 			[self maybeMapPrimitiveFunctionPointerBackToSomethingEvaluable].
+ 		 self dispatchFunctionPointer: primitiveFunctionPointer].
+ 	^retried!

Item was changed:
  ----- Method: StackInterpreter>>slowPrimitiveResponse (in category 'primitive support') -----
  slowPrimitiveResponse
  	"Invoke a normal (non-quick) primitive.
  	 Called under the assumption that primFunctionPointer has been preloaded."
  	| nArgs savedFramePointer savedStackPointer |
  	<inline: true>
  	<var: #savedFramePointer type: #'char *'>
  	<var: #savedStackPointer type: #'char *'>
  	self assert: (objectMemory isOopForwarded: (self stackValue: argumentCount)) not.
  	self assert: objectMemory remapBufferCount = 0.
  	FailImbalancedPrimitives ifTrue:
  		[nArgs := argumentCount.
  		 savedStackPointer := stackPointer.
  		 savedFramePointer := framePointer].
  	self initPrimCall.
  	self dispatchFunctionPointer: primitiveFunctionPointer.
  	self assert: (self maybeLeakCheckExternalPrimCall: newMethod).
+ 	self maybeRetryPrimitiveOnFailure.
- 	self maybeRetryFailureDueToForwarding.
  	self maybeFailForLastObjectOverwrite.
  	(FailImbalancedPrimitives
  	and: [self successful
  	and: [framePointer = savedFramePointer
  	and: [(self isMachineCodeFrame: framePointer) not]]]) ifTrue:"Don't fail if primitive has done something radical, e.g. perform:"
  		[stackPointer ~= (savedStackPointer + (nArgs * objectMemory wordSize)) ifTrue:
  			[self flag: 'Would be nice to make this a message send of e.g. unbalancedPrimitive to the current process or context'.
  			 "This is necessary but insufficient; the result may still have been written to the stack.
  			   At least we'll know something is wrong."
  			 stackPointer := savedStackPointer.
  			 self failUnbalancedPrimitive]].
  	"If we are profiling, take accurate primitive measures"
  	nextProfileTick > 0 ifTrue:
  		[self checkProfileTick: newMethod].
  	^self successful!

Item was changed:
  ----- Method: StackInterpreterPrimitives>>primitiveDoNamedPrimitiveWithArgs (in category 'plugin primitives') -----
  primitiveDoNamedPrimitiveWithArgs
  	"Simulate an primitiveExternalCall invocation (e.g. for the Debugger).  Do not cache anything.
  	 e.g. ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs: arguments"
  	| argumentArray arraySize methodArg methodHeader
  	  moduleName functionName moduleLength functionLength
  	  spec addr primRcvr isArray |
  	<var: #addr declareC: 'void (*addr)()'>
  	argumentArray := self stackTop.
  	methodArg := self stackValue: 2.
  	((objectMemory isArray: argumentArray)
  	 and: [objectMemory isOopCompiledMethod: methodArg]) ifFalse:
  		[^self primitiveFailFor: -2]. "invalid args"
  	arraySize := objectMemory numSlotsOf: argumentArray.
  	(self roomToPushNArgs: arraySize) ifFalse:
  		[^self primitiveFailFor: -2]. "invalid args"
  
  	methodHeader := objectMemory methodHeaderOf: methodArg.
  	(objectMemory literalCountOfMethodHeader: methodHeader) > 2 ifFalse:
  		[^self primitiveFailFor: -3]. "invalid methodArg state"
  	spec := objectMemory fetchPointer: 1 "first literal" ofObject: methodArg.
  	isArray := self isInstanceOfClassArray: spec.
  	(isArray
  	and: [(objectMemory numSlotsOf: spec) = 4
  	and: [(self primitiveIndexOfMethod: methodArg header: methodHeader) = PrimNumberExternalCall]]) ifFalse:
  		[^self primitiveFailFor: -3]. "invalid methodArg state"
  
  	(self argumentCountOfMethodHeader: methodHeader) = arraySize ifFalse:
  		[^self primitiveFailFor: -2]. "invalid args (Array args wrong size)"
  
  	"The function has not been loaded yet. Fetch module and function name."
  	moduleName := objectMemory fetchPointer: 0 ofObject: spec.
  	moduleName = objectMemory nilObject
  		ifTrue: [moduleLength := 0]
  		ifFalse: [self success: (objectMemory isBytes: moduleName).
  				moduleLength := objectMemory lengthOf: moduleName.
  				self cCode: '' inSmalltalk:
  					[ (#('FloatArrayPlugin' 'Matrix2x3Plugin') includes: (self stringOf: moduleName)) "??"
  						ifTrue: [moduleLength := 0  "Cause all of these to fail"]]].
  	functionName := objectMemory fetchPointer: 1 ofObject: spec.
  	self success: (objectMemory isBytes: functionName).
  	functionLength := objectMemory lengthOf: functionName.
  	self successful ifFalse: [^self primitiveFailFor: -3]. "invalid methodArg state"
  
  	addr := self ioLoadExternalFunction: functionName + objectMemory baseHeaderSize
  				OfLength: functionLength
  				FromModule: moduleName + objectMemory baseHeaderSize
  				OfLength: moduleLength.
  	addr = 0 ifTrue:
  		[^self primitiveFailFor: -1]. "could not find function; answer generic failure (see below)"
  
  	"Cannot fail this primitive from now on.  Can only fail the external primitive."
  	tempOop := objectMemory
  						eeInstantiateClassIndex: ClassArrayCompactIndex
  						format: objectMemory arrayFormat
  						numSlots: (objectMemory hasSpurMemoryManagerAPI
  									ifTrue: [5]
  									ifFalse: [4]).
  	objectMemory
  		storePointerUnchecked: 0 ofObject: tempOop withValue: (argumentArray := self popStack);
  		storePointerUnchecked: 1 ofObject: tempOop withValue: (primRcvr := self popStack);
  		storePointerUnchecked: 2 ofObject: tempOop withValue: self popStack; "the method"
  		storePointerUnchecked: 3 ofObject: tempOop withValue: self popStack. "the context receiver"
  	self push: primRcvr. "replace context receiver with actual receiver"
  	argumentCount := arraySize.
  	1 to: arraySize do:
  		[:index| self push: (objectMemory fetchPointer: index - 1 ofObject: argumentArray)].
  	objectMemory hasSpurMemoryManagerAPI
  		ifTrue:
  			[objectMemory storePointerUnchecked: 4 ofObject: tempOop withValue: newMethod.
  			 newMethod := methodArg.
  			 self callExternalPrimitive: addr. "On Spur, sets primitiveFunctionPointer"
+ 			 self maybeRetryPrimitiveOnFailure.
- 			 self maybeRetryFailureDueToForwarding.
  			 newMethod := objectMemory fetchPointer: 4 ofObject: tempOop]
  		ifFalse:
  			[self callExternalPrimitive: addr].
  	self successful ifFalse: "If primitive failed, then restore state for failure code"
  		[self pop: arraySize + 1.
  		 self push: (objectMemory fetchPointer: 3 ofObject: tempOop).
  		 self push: (objectMemory fetchPointer: 2 ofObject: tempOop).
  		 self push: (objectMemory fetchPointer: 1 ofObject: tempOop).
  		 self push: (objectMemory fetchPointer: 0 ofObject: tempOop).
  		 argumentCount := 3.
  		 "Must reset primitiveFunctionPointer for checkForAndFollowForwardedPrimitiveState"
  		 objectMemory hasSpurMemoryManagerAPI ifTrue:
  			[primitiveFunctionPointer := #primitiveDoNamedPrimitiveWithArgs].
  		 "Hack.  A nil prim error code (primErrorCode = 1) is interpreted by the image
  		  as meaning this primitive is not implemented.  So to pass back nil as an error
  		  code we use -1 to indicate generic failure."
  		 primFailCode = 1 ifTrue:
  			[primFailCode := -1]]!

Item was changed:
  ----- Method: StackInterpreterPrimitives>>primitiveExternalCall (in category 'plugin primitives') -----
  primitiveExternalCall
  	"Call an external primitive. External primitive methods first literals are an array of
  		* The module name (String | Symbol)
  		* The function name (String | Symbol)
  		* The session ID (SmallInteger) [OBSOLETE], or in Spur, the accessorDepth (Integer)
  		* The function index (Integer) in the externalPrimitiveTable
  	For fast interpreter dispatch in subsequent invocations the primitiveFunctionPointer
  	in the method cache is rewritten, either to the function itself, or to zero if the external
  	function is not found.   This allows for fast responses as long as the method stays in
  	the cache. The cache rewrite relies on lastMethodCacheProbeWrite which is set in
  	addNewMethodToCache:.
  	Now that the VM flushes function addresses from its tables, the session ID is obsolete,
  	but it is kept for backward compatibility. Also, a failed lookup is reported specially. If a
  	method has been  looked up and not been found, the function address is stored as -1
  	(i.e., the SmallInteger -1 to distinguish from 16rFFFFFFFF which may be returned from
  	lookup), and the primitive fails with PrimErrNotFound."
  	| lit addr moduleName functionName moduleLength functionLength accessorDepth index |
  	<var: #addr declareC: 'void (*addr)()'>
  	
  	"Check for it being a method for primitiveDoPrimitiveWithArgs.
  	 Fetch the first literal of the method; check its an Array of length 4.
  	 Look at the function index in case it has been loaded before"
  	((objectMemory isOopCompiledMethod: newMethod)
  	 and: [(objectMemory literalCountOf: newMethod) > 0
  	 and: [lit := self literal: 0 ofMethod: newMethod.
  		(objectMemory isArray: lit)
  	 and: [(objectMemory numSlotsOf: lit) = 4
  	 and: [index := objectMemory fetchPointer: 3 ofObject: lit.
  		objectMemory isIntegerObject: index]]]]) ifFalse:
  		[^self primitiveFailFor: PrimErrBadMethod].
  
  	index := objectMemory integerValueOf: index.
  	"Check if we have already looked up the function and failed."
  	index < 0 ifTrue:
  		["Function address was not found in this session, 
  		  Void the primitive function."
  		 self rewriteMethodCacheEntryForExternalPrimitiveToFunction: 0.
  		 ^self primitiveFailFor: PrimErrNotFound].
  
  	"Try to call the function directly"
  	(index > 0 and: [index <= MaxExternalPrimitiveTableSize]) ifTrue:
  		[addr := externalPrimitiveTable at: index - 1.
  		 addr ~= 0 ifTrue:
  			[self rewriteMethodCacheEntryForExternalPrimitiveToFunction: (self cCode: 'addr' inSmalltalk: [1000 + index]).
  			 self callExternalPrimitive: addr. "On Spur, sets primitiveFunctionPointer"
+ 			 self maybeRetryPrimitiveOnFailure.
- 			 self maybeRetryFailureDueToForwarding.
  			 ^nil].
  		"if we get here, then an index to the external prim was 
  		kept on the ST side although the underlying prim 
  		table was already flushed"
  		^self primitiveFailFor: PrimErrNamedInternal].
  
  	"Clean up session id and external primitive index"
  	objectMemory storePointerUnchecked: 2 ofObject: lit withValue: ConstZero.
  	objectMemory storePointerUnchecked: 3 ofObject: lit withValue: ConstZero.
  
  	"The function has not been loaded yet. Fetch module and function name."
  	moduleName := objectMemory fetchPointer: 0 ofObject: lit.
  	moduleName = objectMemory nilObject
  		ifTrue: [moduleLength := 0]
  		ifFalse: [(objectMemory isBytes: moduleName) ifFalse:
  					[self primitiveFailFor: PrimErrBadMethod].
  				moduleLength := objectMemory lengthOf: moduleName].
  	functionName := objectMemory fetchPointer: 1 ofObject: lit.
  	(objectMemory isBytes: functionName) ifFalse:
  		[self primitiveFailFor: PrimErrBadMethod].
  	functionLength := objectMemory lengthOf: functionName.
  
  	"Spur needs to know the primitive's accessorDepth which is stored in the third slot of the first literal."
  	objectMemory hasSpurMemoryManagerAPI
  		ifTrue:
  			[addr := self ioLoadExternalFunction: functionName + objectMemory baseHeaderSize
  						OfLength: functionLength
  						FromModule: moduleName + objectMemory baseHeaderSize
  						OfLength: moduleLength
  						AccessorDepthInto: (self addressOf: accessorDepth
  												 put: [:val| accessorDepth := val]).
  			 addr = 0
  				ifTrue: [index := -1]
  				ifFalse: "add the function to the external primitive table"
  					[index := self addToExternalPrimitiveTable: addr.
  					 objectMemory
  						storePointerUnchecked: 2
  						ofObject: lit
  						withValue: (objectMemory integerObjectOf: accessorDepth)]]
  		ifFalse:
  			[addr := self ioLoadExternalFunction: functionName + objectMemory baseHeaderSize
  						OfLength: functionLength
  						FromModule: moduleName + objectMemory baseHeaderSize
  						OfLength: moduleLength.
  			 addr = 0
  				ifTrue: [index := -1]
  				ifFalse: "add the function to the external primitive table"
  					[index := self addToExternalPrimitiveTable: addr]].
  
  	"Store the index (or -1 if failure) back in the literal"
  	objectMemory storePointerUnchecked: 3 ofObject: lit withValue: (objectMemory integerObjectOf: index).
  
  	"If the function has been successfully loaded cache and call it"
  	index >= 0
  		ifTrue:
  			[self rewriteMethodCacheEntryForExternalPrimitiveToFunction: (self cCode: [addr] inSmalltalk: [1000 + index]).
  			 self callExternalPrimitive: addr.
+ 			 self maybeRetryPrimitiveOnFailure]
- 			 self maybeRetryFailureDueToForwarding]
  		ifFalse: "Otherwise void the primitive function and fail"
  			[self rewriteMethodCacheEntryForExternalPrimitiveToFunction: 0.
  			 self assert: (objectMemory fetchPointer: 2 ofObject: lit) = ConstZero.
  			 self primitiveFailFor: PrimErrNotFound]!

Item was added:
+ ----- Method: StackInterpreterSimulator>>mappedPluginEntries (in category 'plugin support') -----
+ mappedPluginEntries
+ 	^mappedPluginEntries!



More information about the Vm-dev mailing list