Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3142.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.3142 Author: eem Time: 21 January 2022, 5:10:43.282744 pm UUID: ecaa84c0-26c9-440b-bc13-59a6e56c5e7c Ancestors: VMMaker.oscog-eem.3141
Change the revised primitiveSuspend to behave clkoser to the original, always answeriong the list the receiver was on if bnlocked. make the newer variant available as a named prim itive (primitiveSuspendV2). AFAICT using primitiveSuspendV2 breaks lots of things (SqueakSSL tests hang, Virtend login hangs). So while the idea is right, in practice answering nil if the receiver is suspended while waiting on a condition variable breaks more things than it fixes.
=============== Diff against VMMaker.oscog-eem.3141 ===============
Item was changed: ----- Method: CoInterpreterPrimitives>>primitiveSuspend (in category 'process primitives') ----- primitiveSuspend "Primitive. Suspend the receiver, aProcess, such that it can be executed again by sending #resume. If the given process is not the active process, take it off its corresponding list. If the list was not its run queue assume it was on some condition variable (Semaphore, Mutex) and back up its pc to the send that invoked the wait state the process entered. Hence when the process resumes + it will reenter the wait state. Answer the list the receiver was previously on, + unless it was the activ eProcess, in which case answer nil." - it will reenter the wait state. Answer the list the receiver was previously on iff - it was not active and not blocked, otherwise answer nil." | process myList myContext ok | process := self stackTop. process = self activeProcess ifTrue: [| inInterpreter | "We're going to switch process, either to an interpreted frame or a machine code frame. To know whether to return or enter machine code we have to know from whence we came. We could have come from the interpreter, either directly or via a machine code primitive. We could have come from machine code. The instructionPointer tells us where from:" self pop: 1 thenPush: objectMemory nilObject. inInterpreter := instructionPointer >= objectMemory startOfMemory. self transferTo: self wakeHighestPriority from: CSSuspend. ^self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]. myList := objectMemory fetchPointer: MyListIndex ofObject: process. myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. ((objectMemory isPointers: myList) and: [(objectMemory numSlotsOf: myList) > LastLinkIndex and: [(objectMemory isContext: myContext) and: [self isResumableContext: myContext]]]) ifFalse: [^self primitiveFailFor: PrimErrBadReceiver]. ok := self removeProcess: process fromList: myList. ok ifFalse: [^self primitiveFailFor: PrimErrOperationFailed]. objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. self assert: RevisedSuspend. (RevisedSuspend + and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) ifTrue: + [self backupContext: myContext toBlockingSendTo: myList]. + self pop: 1 thenPush: myList! - and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) - ifTrue: - [self backupContext: myContext toBlockingSendTo: myList. - self pop: 1 thenPush: objectMemory nilObject] - ifFalse: - [self pop: 1 thenPush: myList]!
Item was added: + ----- Method: CoInterpreterPrimitives>>primitiveSuspendV2 (in category 'process primitives') ----- + primitiveSuspendV2 + "Primitive. Suspend the receiver, aProcess, such that it can be executed again + by sending #resume. If the given process is not the active process, take it off + its corresponding list. If the list was not its run queue assume it was on some + condition variable (Semaphore, Mutex) and back up its pc to the send that + invoked the wait state the process entered. Hence when the process resumes + it will reenter the wait state. Answer the list the receiver was previously on iff + it was not active and not blocked, otherwise answer nil. + c.f. primitiveSuspend, which always answers the list the process was on, if blocked." + <export: true> + | process myList myContext ok | + process := self stackTop. + process = self activeProcess ifTrue: + [| inInterpreter | + "We're going to switch process, either to an interpreted frame or a machine + code frame. To know whether to return or enter machine code we have to + know from whence we came. We could have come from the interpreter, + either directly or via a machine code primitive. We could have come from + machine code. The instructionPointer tells us where from:" + self pop: 1 thenPush: objectMemory nilObject. + inInterpreter := instructionPointer >= objectMemory startOfMemory. + self transferTo: self wakeHighestPriority from: CSSuspend. + ^self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]. + myList := objectMemory fetchPointer: MyListIndex ofObject: process. + myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. + ((objectMemory isPointers: myList) + and: [(objectMemory numSlotsOf: myList) > LastLinkIndex + and: [(objectMemory isContext: myContext) + and: [self isResumableContext: myContext]]]) ifFalse: + [^self primitiveFailFor: PrimErrBadReceiver]. + ok := self removeProcess: process fromList: myList. + ok ifFalse: + [^self primitiveFailFor: PrimErrOperationFailed]. + objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. + self assert: RevisedSuspend. + (RevisedSuspend + and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) + ifTrue: + [self backupContext: myContext toBlockingSendTo: myList. + self pop: 1 thenPush: objectMemory nilObject] + ifFalse: + [self pop: 1 thenPush: myList]!
Item was changed: ----- Method: CogVMSimulator class>>initialize (in category 'class initialization') ----- initialize "These are primitives that alter the state of the stack. They are here simply for assert checking. After invocation the Cogit should not check for the expected stack delta when these primitives succeed, because the stack will usually have been modified." StackAlteringPrimitives := #( primitiveClosureValue primitiveClosureValueWithArgs primitiveClosureValueNoContextSwitch primitiveClone primitiveInstVarAt primitiveSlotAt "because these can cause code compactions..." primitiveEnterCriticalSection primitiveExitCriticalSection primitiveFullClosureValue primitiveFullClosureValueWithArgs primitiveFullClosureValueNoContextSwitch + primitiveSignal primitiveWait primitiveResume primitiveSuspend primitiveSuspendV2 primitiveYield - primitiveSignal primitiveWait primitiveResume primitiveSuspend primitiveYield primitiveExecuteMethodArgsArray primitiveExecuteMethod primitivePerform primitivePerformWithArgs primitivePerformInSuperclass primitiveTerminateTo primitiveStoreStackp primitiveDoPrimitiveWithArgs) asIdentitySet!
Item was changed: ----- Method: StackInterpreterPrimitives>>primitiveSuspend (in category 'process primitives') ----- primitiveSuspend "Primitive. Suspend the receiver, aProcess, such that it can be executed again by sending #resume. If the given process is not the active process, take it off its corresponding list. If the list was not its run queue assume it was on some condition variable (Semaphore, Mutex) and back up its pc to the send that invoked the wait state the process entered. Hence when the process resumes + it will reenter the wait state. Answer the list the receiver was previously on, + unless it was the activ eProcess, in which case answer nil." - it will reenter the wait state. Answer the list the receiver was previously on iff - it was not active and not blocked, otherwise answer nil." | process myList myContext ok | process := self stackTop. process = self activeProcess ifTrue: [self pop: 1 thenPush: objectMemory nilObject. ^self transferTo: self wakeHighestPriority]. myList := objectMemory fetchPointer: MyListIndex ofObject: process. myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. ((objectMemory isPointers: myList) and: [(objectMemory numSlotsOf: myList) > LastLinkIndex and: [(objectMemory isContext: myContext) and: [self isResumableContext: myContext]]]) ifFalse: [^self primitiveFailFor: PrimErrBadReceiver]. ok := self removeProcess: process fromList: myList. ok ifFalse: [^self primitiveFailFor: PrimErrOperationFailed]. objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. self assert: RevisedSuspend. (RevisedSuspend + and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) ifTrue: + [self backupContext: myContext toBlockingSendTo: myList]. + self pop: 1 thenPush: myList! - and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) - ifTrue: - [self backupContext: myContext toBlockingSendTo: myList. - self pop: 1 thenPush: objectMemory nilObject] - ifFalse: - [self pop: 1 thenPush: myList]!
Item was added: + ----- Method: StackInterpreterPrimitives>>primitiveSuspendV2 (in category 'process primitives') ----- + primitiveSuspendV2 + "Primitive. Suspend the receiver, aProcess, such that it can be executed again + by sending #resume. If the given process is not the active process, take it off + its corresponding list. If the list was not its run queue assume it was on some + condition variable (Semaphore, Mutex) and back up its pc to the send that + invoked the wait state the process entered. Hence when the process resumes + it will reenter the wait state. Answer the list the receiver was previously on iff + it was not active and not blocked, otherwise answer nil. + c.f. primitiveSuspend, which always answers the list the process was on, if blocked." + <export: true> + | process myList myContext ok | + process := self stackTop. + process = self activeProcess ifTrue: + [self pop: 1 thenPush: objectMemory nilObject. + ^self transferTo: self wakeHighestPriority]. + myList := objectMemory fetchPointer: MyListIndex ofObject: process. + myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. + ((objectMemory isPointers: myList) + and: [(objectMemory numSlotsOf: myList) > LastLinkIndex + and: [(objectMemory isContext: myContext) + and: [self isResumableContext: myContext]]]) ifFalse: + [^self primitiveFailFor: PrimErrBadReceiver]. + ok := self removeProcess: process fromList: myList. + ok ifFalse: + [^self primitiveFailFor: PrimErrOperationFailed]. + objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. + self assert: RevisedSuspend. + (RevisedSuspend + and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) + ifTrue: + [self backupContext: myContext toBlockingSendTo: myList. + self pop: 1 thenPush: objectMemory nilObject] + ifFalse: + [self pop: 1 thenPush: myList]!
Hi Eliot, I’d like to test it but the build failed for Windows and Mac; not sure what it means… Sorry to hear the current “incorrect” suspend behavior has been exploited more than expected in the existing code. What would you think about having primitiveSuspendV2 return self instead of any list at all – to make all state changing methods answer consistently (like e.g. VW do)… if the answer expected by the “affected” code base has to be provided by primitiveSuspend anyways. Best, Jaromir
From: commits@source.squeak.orgmailto:commits@source.squeak.org Sent: Saturday, January 22, 2022 2:11 To: vm-dev@lists.squeakfoundation.orgmailto:vm-dev@lists.squeakfoundation.org Subject: [Vm-dev] VM Maker: VMMaker.oscog-eem.3142.mcz
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3142.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.3142 Author: eem Time: 21 January 2022, 5:10:43.282744 pm UUID: ecaa84c0-26c9-440b-bc13-59a6e56c5e7c Ancestors: VMMaker.oscog-eem.3141
Change the revised primitiveSuspend to behave clkoser to the original, always answeriong the list the receiver was on if bnlocked. make the newer variant available as a named prim itive (primitiveSuspendV2). AFAICT using primitiveSuspendV2 breaks lots of things (SqueakSSL tests hang, Virtend login hangs). So while the idea is right, in practice answering nil if the receiver is suspended while waiting on a condition variable breaks more things than it fixes.
=============== Diff against VMMaker.oscog-eem.3141 ===============
Item was changed: ----- Method: CoInterpreterPrimitives>>primitiveSuspend (in category 'process primitives') ----- primitiveSuspend "Primitive. Suspend the receiver, aProcess, such that it can be executed again by sending #resume. If the given process is not the active process, take it off its corresponding list. If the list was not its run queue assume it was on some condition variable (Semaphore, Mutex) and back up its pc to the send that invoked the wait state the process entered. Hence when the process resumes + it will reenter the wait state. Answer the list the receiver was previously on, + unless it was the activ eProcess, in which case answer nil." - it will reenter the wait state. Answer the list the receiver was previously on iff - it was not active and not blocked, otherwise answer nil." | process myList myContext ok | process := self stackTop. process = self activeProcess ifTrue: [| inInterpreter | "We're going to switch process, either to an interpreted frame or a machine code frame. To know whether to return or enter machine code we have to know from whence we came. We could have come from the interpreter, either directly or via a machine code primitive. We could have come from machine code. The instructionPointer tells us where from:" self pop: 1 thenPush: objectMemory nilObject. inInterpreter := instructionPointer >= objectMemory startOfMemory. self transferTo: self wakeHighestPriority from: CSSuspend. ^self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]. myList := objectMemory fetchPointer: MyListIndex ofObject: process. myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. ((objectMemory isPointers: myList) and: [(objectMemory numSlotsOf: myList) > LastLinkIndex and: [(objectMemory isContext: myContext) and: [self isResumableContext: myContext]]]) ifFalse: [^self primitiveFailFor: PrimErrBadReceiver]. ok := self removeProcess: process fromList: myList. ok ifFalse: [^self primitiveFailFor: PrimErrOperationFailed]. objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. self assert: RevisedSuspend. (RevisedSuspend + and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) ifTrue: + [self backupContext: myContext toBlockingSendTo: myList]. + self pop: 1 thenPush: myList! - and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) - ifTrue: - [self backupContext: myContext toBlockingSendTo: myList. - self pop: 1 thenPush: objectMemory nilObject] - ifFalse: - [self pop: 1 thenPush: myList]!
Item was added: + ----- Method: CoInterpreterPrimitives>>primitiveSuspendV2 (in category 'process primitives') ----- + primitiveSuspendV2 + "Primitive. Suspend the receiver, aProcess, such that it can be executed again + by sending #resume. If the given process is not the active process, take it off + its corresponding list. If the list was not its run queue assume it was on some + condition variable (Semaphore, Mutex) and back up its pc to the send that + invoked the wait state the process entered. Hence when the process resumes + it will reenter the wait state. Answer the list the receiver was previously on iff + it was not active and not blocked, otherwise answer nil. + c.f. primitiveSuspend, which always answers the list the process was on, if blocked." + <export: true> + | process myList myContext ok | + process := self stackTop. + process = self activeProcess ifTrue: + [| inInterpreter | + "We're going to switch process, either to an interpreted frame or a machine + code frame. To know whether to return or enter machine code we have to + know from whence we came. We could have come from the interpreter, + either directly or via a machine code primitive. We could have come from + machine code. The instructionPointer tells us where from:" + self pop: 1 thenPush: objectMemory nilObject. + inInterpreter := instructionPointer >= objectMemory startOfMemory. + self transferTo: self wakeHighestPriority from: CSSuspend. + ^self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]. + myList := objectMemory fetchPointer: MyListIndex ofObject: process. + myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. + ((objectMemory isPointers: myList) + and: [(objectMemory numSlotsOf: myList) > LastLinkIndex + and: [(objectMemory isContext: myContext) + and: [self isResumableContext: myContext]]]) ifFalse: + [^self primitiveFailFor: PrimErrBadReceiver]. + ok := self removeProcess: process fromList: myList. + ok ifFalse: + [^self primitiveFailFor: PrimErrOperationFailed]. + objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. + self assert: RevisedSuspend. + (RevisedSuspend + and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) + ifTrue: + [self backupContext: myContext toBlockingSendTo: myList. + self pop: 1 thenPush: objectMemory nilObject] + ifFalse: + [self pop: 1 thenPush: myList]!
Item was changed: ----- Method: CogVMSimulator class>>initialize (in category 'class initialization') ----- initialize "These are primitives that alter the state of the stack. They are here simply for assert checking. After invocation the Cogit should not check for the expected stack delta when these primitives succeed, because the stack will usually have been modified." StackAlteringPrimitives := #( primitiveClosureValue primitiveClosureValueWithArgs primitiveClosureValueNoContextSwitch primitiveClone primitiveInstVarAt primitiveSlotAt "because these can cause code compactions..." primitiveEnterCriticalSection primitiveExitCriticalSection primitiveFullClosureValue primitiveFullClosureValueWithArgs primitiveFullClosureValueNoContextSwitch + primitiveSignal primitiveWait primitiveResume primitiveSuspend primitiveSuspendV2 primitiveYield - primitiveSignal primitiveWait primitiveResume primitiveSuspend primitiveYield primitiveExecuteMethodArgsArray primitiveExecuteMethod primitivePerform primitivePerformWithArgs primitivePerformInSuperclass primitiveTerminateTo primitiveStoreStackp primitiveDoPrimitiveWithArgs) asIdentitySet!
Item was changed: ----- Method: StackInterpreterPrimitives>>primitiveSuspend (in category 'process primitives') ----- primitiveSuspend "Primitive. Suspend the receiver, aProcess, such that it can be executed again by sending #resume. If the given process is not the active process, take it off its corresponding list. If the list was not its run queue assume it was on some condition variable (Semaphore, Mutex) and back up its pc to the send that invoked the wait state the process entered. Hence when the process resumes + it will reenter the wait state. Answer the list the receiver was previously on, + unless it was the activ eProcess, in which case answer nil." - it will reenter the wait state. Answer the list the receiver was previously on iff - it was not active and not blocked, otherwise answer nil." | process myList myContext ok | process := self stackTop. process = self activeProcess ifTrue: [self pop: 1 thenPush: objectMemory nilObject. ^self transferTo: self wakeHighestPriority]. myList := objectMemory fetchPointer: MyListIndex ofObject: process. myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. ((objectMemory isPointers: myList) and: [(objectMemory numSlotsOf: myList) > LastLinkIndex and: [(objectMemory isContext: myContext) and: [self isResumableContext: myContext]]]) ifFalse: [^self primitiveFailFor: PrimErrBadReceiver]. ok := self removeProcess: process fromList: myList. ok ifFalse: [^self primitiveFailFor: PrimErrOperationFailed]. objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. self assert: RevisedSuspend. (RevisedSuspend + and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) ifTrue: + [self backupContext: myContext toBlockingSendTo: myList]. + self pop: 1 thenPush: myList! - and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) - ifTrue: - [self backupContext: myContext toBlockingSendTo: myList. - self pop: 1 thenPush: objectMemory nilObject] - ifFalse: - [self pop: 1 thenPush: myList]!
Item was added: + ----- Method: StackInterpreterPrimitives>>primitiveSuspendV2 (in category 'process primitives') ----- + primitiveSuspendV2 + "Primitive. Suspend the receiver, aProcess, such that it can be executed again + by sending #resume. If the given process is not the active process, take it off + its corresponding list. If the list was not its run queue assume it was on some + condition variable (Semaphore, Mutex) and back up its pc to the send that + invoked the wait state the process entered. Hence when the process resumes + it will reenter the wait state. Answer the list the receiver was previously on iff + it was not active and not blocked, otherwise answer nil. + c.f. primitiveSuspend, which always answers the list the process was on, if blocked." + <export: true> + | process myList myContext ok | + process := self stackTop. + process = self activeProcess ifTrue: + [self pop: 1 thenPush: objectMemory nilObject. + ^self transferTo: self wakeHighestPriority]. + myList := objectMemory fetchPointer: MyListIndex ofObject: process. + myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. + ((objectMemory isPointers: myList) + and: [(objectMemory numSlotsOf: myList) > LastLinkIndex + and: [(objectMemory isContext: myContext) + and: [self isResumableContext: myContext]]]) ifFalse: + [^self primitiveFailFor: PrimErrBadReceiver]. + ok := self removeProcess: process fromList: myList. + ok ifFalse: + [^self primitiveFailFor: PrimErrOperationFailed]. + objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. + self assert: RevisedSuspend. + (RevisedSuspend + and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) + ifTrue: + [self backupContext: myContext toBlockingSendTo: myList. + self pop: 1 thenPush: objectMemory nilObject] + ifFalse: + [self pop: 1 thenPush: myList]!
Hi Jaromir --
Sorry to hear the current “incorrect” suspend behavior has been exploited more than expected [...]
It's kind of a rule of thumb. :-) If an existing implementation is older than -- say -- half a year, we must always assume that it as been exploited and design a solution that does not break backwards compatibility, especially if it is something low-level and important such as process management.
Thanks for your onward curiosity in this domain!!
Best, Marcel Am 22.01.2022 12:57:15 schrieb Jaromir Matas mail@jaromir.net:
Hi Eliot,
I’d like to test it but the build failed for Windows and Mac; not sure what it means…
Sorry to hear the current “incorrect” suspend behavior has been exploited more than expected in the existing code. What would you think about having primitiveSuspendV2 return self instead of any list at all – to make all state changing methods answer consistently (like e.g. VW do)… if the answer expected by the “affected” code base has to be provided by primitiveSuspend anyways.
Best,
Jaromir
From: commits@source.squeak.org [mailto:commits@source.squeak.org]
Sent: Saturday, January 22, 2022 2:11
To: vm-dev@lists.squeakfoundation.org [mailto:vm-dev@lists.squeakfoundation.org]
Subject: [Vm-dev] VM Maker: VMMaker.oscog-eem.3142.mcz
Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3142.mcz [http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3142.mcz]
==================== Summary ====================
Name: VMMaker.oscog-eem.3142
Author: eem
Time: 21 January 2022, 5:10:43.282744 pm
UUID: ecaa84c0-26c9-440b-bc13-59a6e56c5e7c
Ancestors: VMMaker.oscog-eem.3141
Change the revised primitiveSuspend to behave clkoser to the original, always answeriong the list the receiver was on if bnlocked. make the newer variant available as a named prim itive (primitiveSuspendV2). AFAICT using primitiveSuspendV2 breaks lots of things (SqueakSSL tests hang, Virtend login hangs). So while the idea is right, in practice answering nil if the receiver is suspended while waiting on a condition variable breaks more things than it fixes.
=============== Diff against VMMaker.oscog-eem.3141 ===============
Item was changed:
----- Method: CoInterpreterPrimitives>>primitiveSuspend (in category 'process primitives') -----
primitiveSuspend
"Primitive. Suspend the receiver, aProcess, such that it can be executed again
by sending #resume. If the given process is not the active process, take it off
its corresponding list. If the list was not its run queue assume it was on some
condition variable (Semaphore, Mutex) and back up its pc to the send that
invoked the wait state the process entered. Hence when the process resumes
+ it will reenter the wait state. Answer the list the receiver was previously on,
+ unless it was the activ eProcess, in which case answer nil."
- it will reenter the wait state. Answer the list the receiver was previously on iff
- it was not active and not blocked, otherwise answer nil."
| process myList myContext ok |
process := self stackTop.
process = self activeProcess ifTrue:
[| inInterpreter |
"We're going to switch process, either to an interpreted frame or a machine
code frame. To know whether to return or enter machine code we have to
know from whence we came. We could have come from the interpreter,
either directly or via a machine code primitive. We could have come from
machine code. The instructionPointer tells us where from:"
self pop: 1 thenPush: objectMemory nilObject.
inInterpreter := instructionPointer >= objectMemory startOfMemory.
self transferTo: self wakeHighestPriority from: CSSuspend.
^self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter].
myList := objectMemory fetchPointer: MyListIndex ofObject: process.
myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process.
((objectMemory isPointers: myList)
and: [(objectMemory numSlotsOf: myList) > LastLinkIndex
and: [(objectMemory isContext: myContext)
and: [self isResumableContext: myContext]]]) ifFalse:
[^self primitiveFailFor: PrimErrBadReceiver].
ok := self removeProcess: process fromList: myList.
ok ifFalse:
[^self primitiveFailFor: PrimErrOperationFailed].
objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject.
self assert: RevisedSuspend.
(RevisedSuspend
+ and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) ifTrue:
+ [self backupContext: myContext toBlockingSendTo: myList].
+ self pop: 1 thenPush: myList!
- and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag])
- ifTrue:
- [self backupContext: myContext toBlockingSendTo: myList.
- self pop: 1 thenPush: objectMemory nilObject]
- ifFalse:
- [self pop: 1 thenPush: myList]!
Item was added:
+ ----- Method: CoInterpreterPrimitives>>primitiveSuspendV2 (in category 'process primitives') -----
+ primitiveSuspendV2
+ "Primitive. Suspend the receiver, aProcess, such that it can be executed again
+ by sending #resume. If the given process is not the active process, take it off
+ its corresponding list. If the list was not its run queue assume it was on some
+ condition variable (Semaphore, Mutex) and back up its pc to the send that
+ invoked the wait state the process entered. Hence when the process resumes
+ it will reenter the wait state. Answer the list the receiver was previously on iff
+ it was not active and not blocked, otherwise answer nil.
+ c.f. primitiveSuspend, which always answers the list the process was on, if blocked."
+ <export: true>
+ | process myList myContext ok |
+ process := self stackTop.
+ process = self activeProcess ifTrue:
+ [| inInterpreter |
+ "We're going to switch process, either to an interpreted frame or a machine
+ code frame. To know whether to return or enter machine code we have to
+ know from whence we came. We could have come from the interpreter,
+ either directly or via a machine code primitive. We could have come from
+ machine code. The instructionPointer tells us where from:"
+ self pop: 1 thenPush: objectMemory nilObject.
+ inInterpreter := instructionPointer >= objectMemory startOfMemory.
+ self transferTo: self wakeHighestPriority from: CSSuspend.
+ ^self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter].
+ myList := objectMemory fetchPointer: MyListIndex ofObject: process.
+ myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process.
+ ((objectMemory isPointers: myList)
+ and: [(objectMemory numSlotsOf: myList) > LastLinkIndex
+ and: [(objectMemory isContext: myContext)
+ and: [self isResumableContext: myContext]]]) ifFalse:
+ [^self primitiveFailFor: PrimErrBadReceiver].
+ ok := self removeProcess: process fromList: myList.
+ ok ifFalse:
+ [^self primitiveFailFor: PrimErrOperationFailed].
+ objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject.
+ self assert: RevisedSuspend.
+ (RevisedSuspend
+ and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag])
+ ifTrue:
+ [self backupContext: myContext toBlockingSendTo: myList.
+ self pop: 1 thenPush: objectMemory nilObject]
+ ifFalse:
+ [self pop: 1 thenPush: myList]!
Item was changed:
----- Method: CogVMSimulator class>>initialize (in category 'class initialization') -----
initialize
"These are primitives that alter the state of the stack. They are here simply for assert checking.
After invocation the Cogit should not check for the expected stack delta when these primitives
succeed, because the stack will usually have been modified."
StackAlteringPrimitives := #( primitiveClosureValue primitiveClosureValueWithArgs primitiveClosureValueNoContextSwitch
primitiveClone primitiveInstVarAt primitiveSlotAt "because these can cause code compactions..."
primitiveEnterCriticalSection primitiveExitCriticalSection
primitiveFullClosureValue primitiveFullClosureValueWithArgs primitiveFullClosureValueNoContextSwitch
+ primitiveSignal primitiveWait primitiveResume primitiveSuspend primitiveSuspendV2 primitiveYield
- primitiveSignal primitiveWait primitiveResume primitiveSuspend primitiveYield
primitiveExecuteMethodArgsArray primitiveExecuteMethod
primitivePerform primitivePerformWithArgs primitivePerformInSuperclass
primitiveTerminateTo primitiveStoreStackp primitiveDoPrimitiveWithArgs) asIdentitySet!
Item was changed:
----- Method: StackInterpreterPrimitives>>primitiveSuspend (in category 'process primitives') -----
primitiveSuspend
"Primitive. Suspend the receiver, aProcess, such that it can be executed again
by sending #resume. If the given process is not the active process, take it off
its corresponding list. If the list was not its run queue assume it was on some
condition variable (Semaphore, Mutex) and back up its pc to the send that
invoked the wait state the process entered. Hence when the process resumes
+ it will reenter the wait state. Answer the list the receiver was previously on,
+ unless it was the activ eProcess, in which case answer nil."
- it will reenter the wait state. Answer the list the receiver was previously on iff
- it was not active and not blocked, otherwise answer nil."
| process myList myContext ok |
process := self stackTop.
process = self activeProcess ifTrue:
[self pop: 1 thenPush: objectMemory nilObject.
^self transferTo: self wakeHighestPriority].
myList := objectMemory fetchPointer: MyListIndex ofObject: process.
myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process.
((objectMemory isPointers: myList)
and: [(objectMemory numSlotsOf: myList) > LastLinkIndex
and: [(objectMemory isContext: myContext)
and: [self isResumableContext: myContext]]]) ifFalse:
[^self primitiveFailFor: PrimErrBadReceiver].
ok := self removeProcess: process fromList: myList.
ok ifFalse:
[^self primitiveFailFor: PrimErrOperationFailed].
objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject.
self assert: RevisedSuspend.
(RevisedSuspend
+ and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag]) ifTrue:
+ [self backupContext: myContext toBlockingSendTo: myList].
+ self pop: 1 thenPush: myList!
- and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag])
- ifTrue:
- [self backupContext: myContext toBlockingSendTo: myList.
- self pop: 1 thenPush: objectMemory nilObject]
- ifFalse:
- [self pop: 1 thenPush: myList]!
Item was added:
+ ----- Method: StackInterpreterPrimitives>>primitiveSuspendV2 (in category 'process primitives') -----
+ primitiveSuspendV2
+ "Primitive. Suspend the receiver, aProcess, such that it can be executed again
+ by sending #resume. If the given process is not the active process, take it off
+ its corresponding list. If the list was not its run queue assume it was on some
+ condition variable (Semaphore, Mutex) and back up its pc to the send that
+ invoked the wait state the process entered. Hence when the process resumes
+ it will reenter the wait state. Answer the list the receiver was previously on iff
+ it was not active and not blocked, otherwise answer nil.
+ c.f. primitiveSuspend, which always answers the list the process was on, if blocked."
+ <export: true>
+ | process myList myContext ok |
+ process := self stackTop.
+ process = self activeProcess ifTrue:
+ [self pop: 1 thenPush: objectMemory nilObject.
+ ^self transferTo: self wakeHighestPriority].
+ myList := objectMemory fetchPointer: MyListIndex ofObject: process.
+ myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process.
+ ((objectMemory isPointers: myList)
+ and: [(objectMemory numSlotsOf: myList) > LastLinkIndex
+ and: [(objectMemory isContext: myContext)
+ and: [self isResumableContext: myContext]]]) ifFalse:
+ [^self primitiveFailFor: PrimErrBadReceiver].
+ ok := self removeProcess: process fromList: myList.
+ ok ifFalse:
+ [^self primitiveFailFor: PrimErrOperationFailed].
+ objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject.
+ self assert: RevisedSuspend.
+ (RevisedSuspend
+ and: [(objectMemory fetchClassTagOfNonImm: myList) ~= classLinkedListClassTag])
+ ifTrue:
+ [self backupContext: myContext toBlockingSendTo: myList.
+ self pop: 1 thenPush: objectMemory nilObject]
+ ifFalse:
+ [self pop: 1 thenPush: myList]!
Hi Jaromir,
On Sat, Jan 22, 2022 at 3:57 AM Jaromir Matas mail@jaromir.net wrote:
Hi Eliot,
I’d like to test it but the build failed for Windows and Mac; not sure what it means…
Sorry to hear the current “incorrect” suspend behavior has been exploited more than expected in the existing code.
It's not the incorrect suspend behaviour that's being exploited, it's the return value of suspend. Existing code appears to depend on the semaphore/mutex a process is blocked on being answered by suspend.
What would you think about having primitiveSuspendV2 return self instead of
any list at all – to make all state changing methods answer consistently (like e.g. VW do)… if the answer expected by the “affected” code base has to be provided by primitiveSuspend anyways.
I don't see this is at all useful. If one isn't interested in the semaphore/mutex return value one does not have to examine it. But there is no way of getting the list once the process has been removed from it other than by having the suspend primitive answer it.
So if one isn't interested in the return value simply write something of the form
aProcess suspend; yourself
But if one *is* interested in the result then the primitive better answer it, nbo?
Best,
Jaromir
*From: *commits@source.squeak.org *Sent: *Saturday, January 22, 2022 2:11 *To: *vm-dev@lists.squeakfoundation.org *Subject: *[Vm-dev] VM Maker: VMMaker.oscog-eem.3142.mcz
Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3142.mcz
==================== Summary ====================
Name: VMMaker.oscog-eem.3142 Author: eem Time: 21 January 2022, 5:10:43.282744 pm UUID: ecaa84c0-26c9-440b-bc13-59a6e56c5e7c Ancestors: VMMaker.oscog-eem.3141
Change the revised primitiveSuspend to behave clkoser to the original, always answeriong the list the receiver was on if bnlocked. make the newer variant available as a named prim itive (primitiveSuspendV2). AFAICT using primitiveSuspendV2 breaks lots of things (SqueakSSL tests hang, Virtend login hangs). So while the idea is right, in practice answering nil if the receiver is suspended while waiting on a condition variable breaks more things than it fixes.
=============== Diff against VMMaker.oscog-eem.3141 ===============
Item was changed: ----- Method: CoInterpreterPrimitives>>primitiveSuspend (in category 'process primitives') ----- primitiveSuspend "Primitive. Suspend the receiver, aProcess, such that it can be executed again by sending #resume. If the given process is not the active process, take it off its corresponding list. If the list was not its run queue assume it was on some condition variable (Semaphore, Mutex) and back up its pc to the send that invoked the wait state the process entered. Hence when the process resumes
it will reenter the wait state. Answer the list the receiver waspreviously on,
unless it was the activ eProcess, in which case answer nil."
it will reenter the wait state. Answer the list the receiver waspreviously on iff
it was not active and not blocked, otherwise answer nil." | process myList myContext ok | process := self stackTop. process = self activeProcess ifTrue: [| inInterpreter | "We're going to switch process, either to an interpretedframe or a machine code frame. To know whether to return or enter machine code we have to know from whence we came. We could have come from the interpreter, either directly or via a machine code primitive. We could have come from machine code. The instructionPointer tells us where from:" self pop: 1 thenPush: objectMemory nilObject. inInterpreter := instructionPointer >= objectMemory startOfMemory. self transferTo: self wakeHighestPriority from: CSSuspend. ^self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]. myList := objectMemory fetchPointer: MyListIndex ofObject: process. myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. ((objectMemory isPointers: myList) and: [(objectMemory numSlotsOf: myList) > LastLinkIndex and: [(objectMemory isContext: myContext) and: [self isResumableContext: myContext]]]) ifFalse: [^self primitiveFailFor: PrimErrBadReceiver]. ok := self removeProcess: process fromList: myList. ok ifFalse: [^self primitiveFailFor: PrimErrOperationFailed]. objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. self assert: RevisedSuspend. (RevisedSuspend
and: [(objectMemory fetchClassTagOfNonImm: myList) ~=classLinkedListClassTag]) ifTrue:
[self backupContext: myContext toBlockingSendTo: myList].self pop: 1 thenPush: myList!
and: [(objectMemory fetchClassTagOfNonImm: myList) ~=classLinkedListClassTag])
ifTrue:[self backupContext: myContext toBlockingSendTo:myList.
self pop: 1 thenPush: objectMemory nilObject]ifFalse:[self pop: 1 thenPush: myList]!Item was added:
- ----- Method: CoInterpreterPrimitives>>primitiveSuspendV2 (in category
'process primitives') -----
- primitiveSuspendV2
"Primitive. Suspend the receiver, aProcess, such that it can beexecuted again
by sending #resume. If the given process is not the activeprocess, take it off
its corresponding list. If the list was not its run queue assumeit was on some
condition variable (Semaphore, Mutex) and back up its pc to thesend that
invoked the wait state the process entered. Hence when theprocess resumes
it will reenter the wait state. Answer the list the receiver waspreviously on iff
it was not active and not blocked, otherwise answer nil.c.f. primitiveSuspend, which always answers the list the processwas on, if blocked."
<export: true>| process myList myContext ok |process := self stackTop.process = self activeProcess ifTrue:[| inInterpreter |"We're going to switch process, either to an interpretedframe or a machine
code frame. To know whether to return or enter machinecode we have to
know from whence we came. We could have come from theinterpreter,
either directly or via a machine code primitive. Wecould have come from
machine code. The instructionPointer tells us wherefrom:"
self pop: 1 thenPush: objectMemory nilObject.inInterpreter := instructionPointer >= objectMemorystartOfMemory.
self transferTo: self wakeHighestPriority from: CSSuspend.^selfforProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter].
myList := objectMemory fetchPointer: MyListIndex ofObject:process.
myContext := objectMemory fetchPointer: SuspendedContextIndexofObject: process.
((objectMemory isPointers: myList)and: [(objectMemory numSlotsOf: myList) > LastLinkIndexand: [(objectMemory isContext: myContext)and: [self isResumableContext: myContext]]]) ifFalse:[^self primitiveFailFor: PrimErrBadReceiver].ok := self removeProcess: process fromList: myList.ok ifFalse:[^self primitiveFailFor: PrimErrOperationFailed].objectMemory storePointerUnchecked: MyListIndex ofObject: processwithValue: objectMemory nilObject.
self assert: RevisedSuspend.(RevisedSuspendand: [(objectMemory fetchClassTagOfNonImm: myList) ~=classLinkedListClassTag])
ifTrue:[self backupContext: myContext toBlockingSendTo:myList.
self pop: 1 thenPush: objectMemory nilObject]ifFalse:[self pop: 1 thenPush: myList]!Item was changed: ----- Method: CogVMSimulator class>>initialize (in category 'class initialization') ----- initialize "These are primitives that alter the state of the stack. They are here simply for assert checking. After invocation the Cogit should not check for the expected stack delta when these primitives succeed, because the stack will usually have been modified." StackAlteringPrimitives := #( primitiveClosureValue primitiveClosureValueWithArgs primitiveClosureValueNoContextSwitch
primitiveClone primitiveInstVarAt primitiveSlotAt "because these can cause code compactions..."
primitiveEnterCriticalSection primitiveExitCriticalSection
primitiveFullClosureValue primitiveFullClosureValueWithArgs primitiveFullClosureValueNoContextSwitch
primitiveSignal primitiveWait primitiveResume primitiveSuspend primitiveSuspendV2 primitiveYield
primitiveSignal primitiveWait primitiveResume primitiveSuspend primitiveYield
primitiveExecuteMethodArgsArray primitiveExecuteMethod
primitivePerform primitivePerformWithArgs primitivePerformInSuperclass
primitiveTerminateTo primitiveStoreStackp primitiveDoPrimitiveWithArgs) asIdentitySet!
Item was changed: ----- Method: StackInterpreterPrimitives>>primitiveSuspend (in category 'process primitives') ----- primitiveSuspend "Primitive. Suspend the receiver, aProcess, such that it can be executed again by sending #resume. If the given process is not the active process, take it off its corresponding list. If the list was not its run queue assume it was on some condition variable (Semaphore, Mutex) and back up its pc to the send that invoked the wait state the process entered. Hence when the process resumes
it will reenter the wait state. Answer the list the receiver waspreviously on,
unless it was the activ eProcess, in which case answer nil."
it will reenter the wait state. Answer the list the receiver waspreviously on iff
it was not active and not blocked, otherwise answer nil." | process myList myContext ok | process := self stackTop. process = self activeProcess ifTrue: [self pop: 1 thenPush: objectMemory nilObject. ^self transferTo: self wakeHighestPriority]. myList := objectMemory fetchPointer: MyListIndex ofObject:process. myContext := objectMemory fetchPointer: SuspendedContextIndex ofObject: process. ((objectMemory isPointers: myList) and: [(objectMemory numSlotsOf: myList) > LastLinkIndex and: [(objectMemory isContext: myContext) and: [self isResumableContext: myContext]]]) ifFalse: [^self primitiveFailFor: PrimErrBadReceiver]. ok := self removeProcess: process fromList: myList. ok ifFalse: [^self primitiveFailFor: PrimErrOperationFailed]. objectMemory storePointerUnchecked: MyListIndex ofObject: process withValue: objectMemory nilObject. self assert: RevisedSuspend. (RevisedSuspend
and: [(objectMemory fetchClassTagOfNonImm: myList) ~=classLinkedListClassTag]) ifTrue:
[self backupContext: myContext toBlockingSendTo: myList].self pop: 1 thenPush: myList!
and: [(objectMemory fetchClassTagOfNonImm: myList) ~=classLinkedListClassTag])
ifTrue:[self backupContext: myContext toBlockingSendTo:myList.
self pop: 1 thenPush: objectMemory nilObject]ifFalse:[self pop: 1 thenPush: myList]!Item was added:
- ----- Method: StackInterpreterPrimitives>>primitiveSuspendV2 (in
category 'process primitives') -----
- primitiveSuspendV2
"Primitive. Suspend the receiver, aProcess, such that it can beexecuted again
by sending #resume. If the given process is not the activeprocess, take it off
its corresponding list. If the list was not its run queue assumeit was on some
condition variable (Semaphore, Mutex) and back up its pc to thesend that
invoked the wait state the process entered. Hence when theprocess resumes
it will reenter the wait state. Answer the list the receiver waspreviously on iff
it was not active and not blocked, otherwise answer nil.c.f. primitiveSuspend, which always answers the list the processwas on, if blocked."
<export: true>| process myList myContext ok |process := self stackTop.process = self activeProcess ifTrue:[self pop: 1 thenPush: objectMemory nilObject.^self transferTo: self wakeHighestPriority].myList := objectMemory fetchPointer: MyListIndex ofObject:process.
myContext := objectMemory fetchPointer: SuspendedContextIndexofObject: process.
((objectMemory isPointers: myList)and: [(objectMemory numSlotsOf: myList) > LastLinkIndexand: [(objectMemory isContext: myContext)and: [self isResumableContext: myContext]]]) ifFalse:[^self primitiveFailFor: PrimErrBadReceiver].ok := self removeProcess: process fromList: myList.ok ifFalse:[^self primitiveFailFor: PrimErrOperationFailed].objectMemory storePointerUnchecked: MyListIndex ofObject: processwithValue: objectMemory nilObject.
self assert: RevisedSuspend.(RevisedSuspendand: [(objectMemory fetchClassTagOfNonImm: myList) ~=classLinkedListClassTag])
ifTrue:[self backupContext: myContext toBlockingSendTo:myList.
self pop: 1 thenPush: objectMemory nilObject]ifFalse:[self pop: 1 thenPush: myList]!
Hi Eliot,
Hi Jaromir,
On Sat, Jan 22, 2022 at 3:57 AM Jaromir Matas <mail at jaromir.net> wrote:
Hi Eliot,
I’d like to test it but the build failed for Windows and Mac; not sure what it means…
Sorry to hear the current “incorrect” suspend behavior has been exploited more than expected in the existing code.
It's not the incorrect suspend behaviour that's being exploited, it's the return value of suspend. Existing code appears to depend on the semaphore/mutex a process is blocked on being answered by suspend.
Yes, that's what I meant, actually :) I understand knowing the list was vital to e.g. recognize whether the process was blocked or runnable (#releaseCriticalSection).
What would you think about having primitiveSuspendV2 return self instead of
any list at all – to make all state changing methods answer consistently (like e.g. VW do)… if the answer expected by the “affected” code base has to be provided by primitiveSuspend anyways.
I don't see this is at all useful. If one isn't interested in the semaphore/mutex return value one does not have to examine it. But there is no way of getting the list once the process has been removed from it other than by having the suspend primitive answer it.
Well yes, it surely is more useful to return something in addition to the powerful side effect :) I was just wondering if it's not a "temptation" to be better avoided. I suggested in another post the list is still available on the suspendedContext's stack should someone really want it. But I'm just inquiring here to satisfy my curiosity, sorry.
So if one isn't interested in the return value simply write something of the form
aProcess suspend; yourselfBut if one *is* interested in the result then the primitive better answer it, nbo?
Oh, if there's a use case even with the new primitiveSuspendV2 then no doubt.
Thanks again for sharing your thoughts!
Best,
Jaromir
________________________________ From: Vm-dev vm-dev-bounces@lists.squeakfoundation.org on behalf of Eliot Miranda eliot.miranda@gmail.com Sent: Sunday, January 23, 2022 12:33:08 AM To: Open Smalltalk Virtual Machine Development Discussion vm-dev@lists.squeakfoundation.org Subject: Re: [Vm-dev] VM Maker: VMMaker.oscog-eem.3142.mcz
vm-dev@lists.squeakfoundation.org
-
commits＠source.squeak.org -
Eliot Miranda -
Jaromir Matas -
Marcel Taeumel