On Sun, May 22, 2016 at 2:15 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
On Fri, May 20, 2016 at 7:52 PM, Ben Coman btc@openinworld.com wrote:
On Sat, May 21, 2016 at 4:36 AM, Clément Bera bera.clement@gmail.com wrote:
On Fri, May 20, 2016 at 7:51 PM, Ben Coman btc@openinworld.com wrote:
On Fri, May 20, 2016 at 9:25 PM, Clément Bera bera.clement@gmail.com wrote:
On Thu, May 19, 2016 at 3:42 PM, Ben Coman btc@openinworld.com wrote:
There is a better way of solving this, and that is to use a pragma to identify a method that contains such a suspension point, and have the process terminate code look for the pragma and act accordingly. For example, the pragma could have a terminate action, sent to the receiver with the context as argument, e.g.
Mutex>>critical: mutuallyExcludedBlock <onTerminate: #ensureMutexUnlockedInCritical:> ^lock waitAcquire ifNil: mutuallyExcludedBlock ifNotNil:[ mutuallyExcludedBlock ensure: [lock release] ]
(and here I'm guessing...)
Mutex>> ensureMutexUnlockedInCritical: aContext "long-winded comment explaining the corner case, referencing tests, etc, etc and how it is solved on terminate buy this method" (aContext pc = aContext initialPC and: [self inTheCorner]) ifTrue: [self doTheRightThingTM]
So on terminate the stack is walked (it is anyway) looking for unwinds or onTerminate: markers. Any onTerminate: markers are evaluated, and the corner case is solved. The pragma approach also allows for visibility in the code.
I think this general < onTerminate: > pragma might be useful, but I'd like to keep it in the back pocket for the moment while I explore another idea for primitive retry after a process resumes.
I still have a concern that #primitiveOwnedLockWaitAcquire sleeps at the bottom of the primitive, thus if the sleeping process is resumed, it continues into the critical section without having gained the mutex, which seems a bit fragile. Retrying #primitiveOwnedLockWaitAcquire immediately after waking would effectively have the process sleep at the top of the primitive, and *not*proceed until it *really* holds the lock..
So I'm thinking out loud here to formulate my thoughts, and in case there is some major impediment you can help me fail fast...
One possibility is putting "self maybeRetryFailureAfterWaking" in #slowPrimitiveResponse, similar to maybeRetryFailureDueToForwarding and (guessing) maybeRetryFailureDueToLowMemory. The difficulty seems to be that process's primFailCode doesn't hold across process suspension(??).
As an aside, it seems fragile that IIUC it is possible for primFailCode to be set by one process, which if then suspended will carry over to fail the new active process. Perhaps somewhere like externalSetStackPageAndPointersForSuspendedContextOfProcess: should zero primFailCode.
Anyway... I thought one way to retain primFailCode across process suspension might be to push it to the stack in #transferTo: and pop primFailCode from the stack in externalSetStackPageAndPointersForSuspendedContextOfProcess: except that I see that method called from a few places, so messing with the stack here is probably a bad idea.
Another way might be for Process to get an additional instance variable 'suspendedPrimitiveFailCode' which again could be set in #transferTo: (or even more specifically only in #primitiveOwnedLockWaitAcquire). That is,
#transferTo: might have...
oldProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched. objectMemory storePointer: SuspendedPrimitiveFailCodeIndex ofObject: oldProc withValue: primFailCode ... primFailCode := objectMemory fetchPointer: SuspendedPrimitiveFailCodeIndex ofObject: newProc. self externalSetStackPageAndPointersForSuspendedContextOfProcess: newProc.
The additional advantage here might be that the saving and restoring of primFailCode is localised to one method. I'm hoping that change (plus similar in Cog) might be sufficient to facilitate behaviour like this...
process 1 1. invokes slowPrimitiveResponse 2. dispatches to primitiveOwnedLockWaitAcquire 3. calls primitiveFailFor: PrimErrRetryAfterWaking 4. primitiveFail saved into Process object 5. process goes to sleep
6. later after process 1 woken 7. primitiveFail restored from Process object 8. returns to slowPrimitiveResponse 9. if PrimErrRetryAfterWaking 10. dispatch to primitiveOwnedLockWaitAcquire (goto step 2)
Maybe there would need to be a step 9a to checkForInterrupts to avoid too tight a loop locking the image, but maybe this is already done somewhere in the suspend/resume process.
So I'm now going to try coding the second way in the StackVM, and then look at how it might be done in Cog. All feedback appreciated.
cheers -ben
On Fri, Jun 3, 2016 at 12:52 PM, Ben Coman btc@openinworld.com wrote:
On Sun, May 22, 2016 at 2:15 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
On Fri, May 20, 2016 at 7:52 PM, Ben Coman btc@openinworld.com wrote:
On Sat, May 21, 2016 at 4:36 AM, Clément Bera bera.clement@gmail.com wrote:
On Fri, May 20, 2016 at 7:51 PM, Ben Coman btc@openinworld.com wrote:
On Fri, May 20, 2016 at 9:25 PM, Clément Bera bera.clement@gmail.com wrote:
On Thu, May 19, 2016 at 3:42 PM, Ben Coman btc@openinworld.com wrote:
There is a better way of solving this, and that is to use a pragma to identify a method that contains such a suspension point, and have the process terminate code look for the pragma and act accordingly. For example, the pragma could have a terminate action, sent to the receiver with the context as argument, e.g.
Mutex>>critical: mutuallyExcludedBlock <onTerminate: #ensureMutexUnlockedInCritical:> ^lock waitAcquire ifNil: mutuallyExcludedBlock ifNotNil:[ mutuallyExcludedBlock ensure: [lock release] ]
(and here I'm guessing...)
Mutex>> ensureMutexUnlockedInCritical: aContext "long-winded comment explaining the corner case, referencing tests, etc, etc and how it is solved on terminate buy this method" (aContext pc = aContext initialPC and: [self inTheCorner]) ifTrue: [self doTheRightThingTM]
So on terminate the stack is walked (it is anyway) looking for unwinds or onTerminate: markers. Any onTerminate: markers are evaluated, and the corner case is solved. The pragma approach also allows for visibility in the code.
I think this general < onTerminate: > pragma might be useful, but I'd like to keep it in the back pocket for the moment while I explore another idea for primitive retry after a process resumes.
I still have a concern that #primitiveOwnedLockWaitAcquire sleeps at the bottom of the primitive, thus if the sleeping process is resumed, it continues into the critical section without having gained the mutex, which seems a bit fragile. Retrying #primitiveOwnedLockWaitAcquire immediately after waking would effectively have the process sleep at the top of the primitive, and *not*proceed until it *really* holds the lock..
So I'm thinking out loud here to formulate my thoughts, and in case there is some major impediment you can help me fail fast...
One possibility is putting "self maybeRetryFailureAfterWaking" in #slowPrimitiveResponse, similar to maybeRetryFailureDueToForwarding and (guessing) maybeRetryFailureDueToLowMemory. The difficulty seems to be that process's primFailCode doesn't hold across process suspension(??).
As an aside, it seems fragile that IIUC it is possible for primFailCode to be set by one process, which if then suspended will carry over to fail the new active process. Perhaps somewhere like externalSetStackPageAndPointersForSuspendedContextOfProcess: should zero primFailCode.
Anyway... I thought one way to retain primFailCode across process suspension might be to push it to the stack in #transferTo: and pop primFailCode from the stack in externalSetStackPageAndPointersForSuspendedContextOfProcess: except that I see that method called from a few places, so messing with the stack here is probably a bad idea.
Another way might be for Process to get an additional instance variable 'suspendedPrimitiveFailCode' which again could be set in #transferTo: (or even more specifically only in #primitiveOwnedLockWaitAcquire). That is,
#transferTo: might have...
oldProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched. objectMemory storePointer: SuspendedPrimitiveFailCodeIndex ofObject: oldProc withValue: primFailCode ... primFailCode := objectMemory fetchPointer: SuspendedPrimitiveFailCodeIndex ofObject: newProc. self externalSetStackPageAndPointersForSuspendedContextOfProcess: newProc.
Whoops, for a start, that needed to be... objectMemory storeInteger: SuspendedPrimitiveFailCodeIndex ofObject: oldProc withValue: primFailCode. primFailCode := objectMemory fetchInteger: SuspendedPrimitiveFailCodeIndex ofObject: newProc.
The additional advantage here might be that the saving and restoring of primFailCode is localised to one method. I'm hoping that change (plus similar in Cog) might be sufficient to facilitate behaviour like this...
process 1
invokes slowPrimitiveResponse
dispatches to primitiveOwnedLockWaitAcquire
calls primitiveFailFor: PrimErrRetryAfterWaking
primitiveFail saved into Process object
process goes to sleep
later after process 1 woken
primitiveFail restored from Process object
returns to slowPrimitiveResponse
if PrimErrRetryAfterWaking
dispatch to primitiveOwnedLockWaitAcquire (goto step 2)
Maybe there would need to be a step 9a to checkForInterrupts to avoid too tight a loop locking the image, but maybe this is already done somewhere in the suspend/resume process.
So I'm now going to try coding the second way in the StackVM, and then look at how it might be done in Cog. All feedback appreciated.
cheers -ben
On Fri, Jun 3, 2016 at 3:08 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jun 3, 2016 at 12:52 PM, Ben Coman btc@openinworld.com wrote:
On Sun, May 22, 2016 at 2:15 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
On Fri, May 20, 2016 at 7:52 PM, Ben Coman btc@openinworld.com wrote:
On Sat, May 21, 2016 at 4:36 AM, Clément Bera bera.clement@gmail.com wrote:
On Fri, May 20, 2016 at 7:51 PM, Ben Coman btc@openinworld.com wrote:
On Fri, May 20, 2016 at 9:25 PM, Clément Bera bera.clement@gmail.com wrote: > On Thu, May 19, 2016 at 3:42 PM, Ben Coman btc@openinworld.com wrote:
There is a better way of solving this, and that is to use a pragma to identify a method that contains such a suspension point, and have the process terminate code look for the pragma and act accordingly. For example, the pragma could have a terminate action, sent to the receiver with the context as argument, e.g.
Mutex>>critical: mutuallyExcludedBlock <onTerminate: #ensureMutexUnlockedInCritical:> ^lock waitAcquire ifNil: mutuallyExcludedBlock ifNotNil:[ mutuallyExcludedBlock ensure: [lock release] ]
(and here I'm guessing...)
Mutex>> ensureMutexUnlockedInCritical: aContext "long-winded comment explaining the corner case, referencing tests, etc, etc and how it is solved on terminate buy this method" (aContext pc = aContext initialPC and: [self inTheCorner]) ifTrue: [self doTheRightThingTM]
So on terminate the stack is walked (it is anyway) looking for unwinds or onTerminate: markers. Any onTerminate: markers are evaluated, and the corner case is solved. The pragma approach also allows for visibility in the code.
I think this general < onTerminate: > pragma might be useful, but I'd like to keep it in the back pocket for the moment while I explore another idea for primitive retry after a process resumes.
I still have a concern that #primitiveOwnedLockWaitAcquire sleeps at the bottom of the primitive, thus if the sleeping process is resumed, it continues into the critical section without having gained the mutex, which seems a bit fragile. Retrying #primitiveOwnedLockWaitAcquire immediately after waking would effectively have the process sleep at the top of the primitive, and *not*proceed until it *really* holds the lock..
So I'm thinking out loud here to formulate my thoughts, and in case there is some major impediment you can help me fail fast...
One possibility is putting "self maybeRetryFailureAfterWaking" in #slowPrimitiveResponse, similar to maybeRetryFailureDueToForwarding and (guessing) maybeRetryFailureDueToLowMemory. The difficulty seems to be that process's primFailCode doesn't hold across process suspension(??).
As an aside, it seems fragile that IIUC it is possible for primFailCode to be set by one process, which if then suspended will carry over to fail the new active process. Perhaps somewhere like externalSetStackPageAndPointersForSuspendedContextOfProcess: should zero primFailCode.
Anyway... I thought one way to retain primFailCode across process suspension might be to push it to the stack in #transferTo: and pop primFailCode from the stack in externalSetStackPageAndPointersForSuspendedContextOfProcess: except that I see that method called from a few places, so messing with the stack here is probably a bad idea.
Another way might be for Process to get an additional instance variable 'suspendedPrimitiveFailCode' which again could be set in #transferTo: (or even more specifically only in #primitiveOwnedLockWaitAcquire). That is,
#transferTo: might have...
oldProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched. objectMemory storePointer: SuspendedPrimitiveFailCodeIndex ofObject: oldProc withValue: primFailCode ... primFailCode := objectMemory fetchPointer: SuspendedPrimitiveFailCodeIndex ofObject: newProc. self externalSetStackPageAndPointersForSuspendedContextOfProcess: newProc.
Whoops, for a start, that needed to be (*1*)... objectMemory storeInteger: SuspendedPrimitiveFailCodeIndex ofObject: oldProc withValue: primFailCode. primFailCode := objectMemory fetchInteger: SuspendedPrimitiveFailCodeIndex ofObject: newProc.
The additional advantage here might be that the saving and restoring of primFailCode is localised to one method. I'm hoping that change (plus similar in Cog) might be sufficient to facilitate behaviour like this...
process 1
invokes slowPrimitiveResponse
dispatches to primitiveOwnedLockWaitAcquire
calls primitiveFailFor: PrimErrRetryAfterWaking
primitiveFail saved into Process object
process goes to sleep
later after process 1 woken
primitiveFail restored from Process object
returns to slowPrimitiveResponse
if PrimErrRetryAfterWaking
dispatch to primitiveOwnedLockWaitAcquire (goto step 2)
Maybe there would need to be a step 9a to checkForInterrupts to avoid too tight a loop locking the image, but maybe this is already done somewhere in the suspend/resume process.
So I'm now going to try coding the second way in the StackVM, and then look at how it might be done in Cog. All feedback appreciated.
cheers -ben
My my experiment to better understand this is running the simulator thus... | cos | cos := StackInterpreterSimulator newWithOptions: #(ObjectMemory Spur32BitMemoryManager). cos desiredNumStackPages: 8. cos openOn: 'ownedlock-reader.image'. cos openAsMorph; run
and at the simulator's REPL input box running... o := OwnedLock new. o experiment1. !
where OwnedLock>>experiment1 is... | result | result := OrderedCollection new: 20. result myAdd: 0.
[result myAdd: 11. [result myAdd: 12. self experimentSuccessAndSleep] on: Error do: [result myAdd: 13]. result myAdd: 14] forkAt: 72.
[result myAdd: 21. [result myAdd: 22. self experimentFailAndWakeOther] on: Error do: [result myAdd: 23]. result myAdd: 24] forkAt: 71.
result myAdd: 8. self experimentSuccessAndWakeOther. result myAdd: 9. ^result.
The current VM produces a result of... OrderedCollection(0 11 12 21 22 13 14 24 8 9)
but I'm hoping to get something like... OrderedCollection(0 11 12 21 22 14 23 24 8 9)
Where the primitives are...
primitiveExperimentSuccessAndSleep | ownedLock activeProc | ownedLock := self stackTop. activeProc := self activeProcess. self addLastLink: activeProc toList: ownedLock. self transferTo: self wakeHighestPriority.
primitiveExperimentFailAndWakeOther | ownedLock waitingProcess | ownedLock := self stackTop. self primitiveFailFor: 42. (self isEmptyList: ownedLock) ifFalse: [waitingProcess := self removeFirstLinkOfList: ownedLock. self resume: waitingProcess preemptedYieldingIf: preemptionYields]
primitiveExperimentSuccessAndWakeOther | ownedLock waitingProcess | ownedLock := self stackTop. "rcvr" (self isEmptyList: ownedLock) ifFalse: [waitingProcess := self removeFirstLinkOfList: ownedLock. self resume: waitingProcess preemptedYieldingIf: preemptionYields]
On Fri, Jun 3, 2016 at 6:33 PM, Ben Coman btc@openinworld.com wrote:
where OwnedLock>>experiment1 is... | result | result := OrderedCollection new: 20. result myAdd: 0.
btw, OrderedCollection>>myAdd: is direct copy of OrderderedCollection>>add: just so [set break selector...] can distinguish these from every send of add: in the image.
The current VM produces a result of... OrderedCollection(0 11 12 21 22 13 14 24 8 9)
but I'm hoping to get something like... OrderedCollection(0 11 12 21 22 14 23 24 8 9)
After loading my 'ownedlock-reader.image' with an unchaged VM, then adding these two lines to transferTo:
objectMemory storeInteger: SuspendedPrimitiveFailCodeIndex ofObject: oldProc withValue: primFailCode. primFailCode := objectMemory fetchInteger: SuspendedPrimitiveFailCodeIndex ofObject: newProc.
then executing... OwnedLock new experiment1 ! I get an AssertionFailure in...
Spur32BitMMLESimulator>>fetchPointer: fieldIndex ofObject: objOop self assert: (self isForwarded: objOop) not. self assert: (fieldIndex >= 0 and: [fieldIndex < (self numSlotsOfAny: objOop) or: [fieldIndex = 0 "forwarders and free objs"]]). ^super fetchPointer: fieldIndex ofObject: objOop
where... objOop ==> 16r2D2490: a(n) OwnedLock 16r41A280 nil 16r41A280 nil 16r41A280 nil
fieldIndex ==> 3
self numSlotsOfAny: objOop ==> 3
One thing I'm curious about is why the " < " and not a " <= " ? super fetchPointer: fieldIndex ofObject: objOop successfully returns... 16r41A280: a(n) UndefinedObject
btw, here is the call stack... -16r106C [] in OwnedLock>experiment1 16r2D2490: a(n) OwnedLock -16r104C BlockClosure>on:do: 16r2D29C8: a(n) BlockClosure -16r1024 [] in OwnedLock>experiment1 16r2D2490: a(n) OwnedLock -16r1008 [] in BlockClosure>newProcess 16r2D27E8: a(n) BlockClosure
and the ext head frame... -16r106C [] in OwnedLock>experiment1 16r2D2490: a(n) OwnedLock -16r1064/3559: rcvr/clsr: 16r2D29C8 a BlockClosure -16r1068/3558:cllr ip/ctxt: 16rA3F922=10746146 -16r106C/3557: saved fp: -16r104C=-4172 -16r1070/3556: method: 16r81EF10 a CompiledMethod -16r1074/3555: flags: 16r1010001=16842753 numArgs: 0 hasContext: true isBlock: true -16r1078/3554: context: 16r2D29F8=2959864 -16r107C/3553: receiver: 16r2D2490 an OwnedLock -16r1080/3552: temp/stck: 16r2D24A8 an OrderedCollection -16r1084/3551: temp/stck: 16r2D2490 an OwnedLock
and the int head frame... -16r108C OwnedLock(Process)>suspend 16r2D2490: a(n) OwnedLock -16r1084/3551: rcvr/clsr: 16r2D2490 an OwnedLock -16r1088/3550:cllr ip/ctxt: 16r81EFBB=8515515 -16r108C/3549: saved fp: -16r106C=-4204 -16r1090/3548: method: 16r10DF5B0 a CompiledMethod -16r1094/3547: flags: 16r1=1 numArgs: 0 hasContext: false isBlock: false -16r1098/3546: context: 16r41A280=4301440 -16r109C/3545: receiver: 16r2D2490 an OwnedLock -16r10A0/3544: temp/stck: 16r41A280 nil
What is the difference between ext and int head frames?
One thing I find curious here is "OwnedLock(Process)>suspend". Does this indicate that #suspend was sent to an OwnedLock and looked up the hierarchy to find the method in Process? But OwnedLock doesn't inherit from Process, it inherits from LinkedList.
cheers -ben
Hi Ben,
On Fri, Jun 3, 2016 at 2:30 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jun 3, 2016 at 6:33 PM, Ben Coman btc@openinworld.com wrote:
where OwnedLock>>experiment1 is... | result | result := OrderedCollection new: 20. result myAdd: 0.
btw, OrderedCollection>>myAdd: is direct copy of OrderderedCollection>>add: just so [set break selector...] can distinguish these from every send of add: in the image.
The current VM produces a result of... OrderedCollection(0 11 12 21 22 13 14 24 8 9)
but I'm hoping to get something like... OrderedCollection(0 11 12 21 22 14 23 24 8 9)
After loading my 'ownedlock-reader.image' with an unchaged VM, then adding these two lines to transferTo:
objectMemory storeInteger: SuspendedPrimitiveFailCodeIndex ofObject: oldProc withValue: primFailCode. primFailCode := objectMemory fetchInteger: SuspendedPrimitiveFailCodeIndex ofObject: newProc.
then executing... OwnedLock new experiment1 ! I get an AssertionFailure in...
Spur32BitMMLESimulator>>fetchPointer: fieldIndex ofObject: objOop self assert: (self isForwarded: objOop) not. self assert: (fieldIndex >= 0 and: [fieldIndex < (self numSlotsOfAny: objOop) or: [fieldIndex = 0 "forwarders and free objs"]]). ^super fetchPointer: fieldIndex ofObject: objOop
where... objOop ==> 16r2D2490: a(n) OwnedLock 16r41A280 nil 16r41A280 nil 16r41A280 nil
fieldIndex ==> 3
self numSlotsOfAny: objOop ==> 3
One thing I'm curious about is why the " < " and not a " <= " ? super fetchPointer: fieldIndex ofObject: objOop successfully returns... 16r41A280: a(n) UndefinedObject
Can you print the class definition of OwnedLock ?
Maybe I am wrong but it looks like it's because it's 0-based so the fieldIndex can be between 0 and 2 if numSlots is 3. That would imply a bytecode compiler bug if that object is fixed-sized.
When you try: objectMemory storePointer: 3 ofObject: objOop withValue: objOop
And then print objOop with printOop, is the 3rd field edited ? If not, what if you put 2 as 1st argument ?
btw, here is the call stack... -16r106C [] in OwnedLock>experiment1 16r2D2490: a(n) OwnedLock -16r104C BlockClosure>on:do: 16r2D29C8: a(n) BlockClosure -16r1024 [] in OwnedLock>experiment1 16r2D2490: a(n) OwnedLock -16r1008 [] in BlockClosure>newProcess 16r2D27E8: a(n) BlockClosure
and the ext head frame... -16r106C [] in OwnedLock>experiment1 16r2D2490: a(n) OwnedLock -16r1064/3559: rcvr/clsr: 16r2D29C8 a BlockClosure -16r1068/3558:cllr ip/ctxt: 16rA3F922=10746146 -16r106C/3557: saved fp: -16r104C=-4172 -16r1070/3556: method: 16r81EF10 a CompiledMethod -16r1074/3555: flags: 16r1010001=16842753 numArgs: 0 hasContext: true isBlock: true -16r1078/3554: context: 16r2D29F8=2959864 -16r107C/3553: receiver: 16r2D2490 an OwnedLock -16r1080/3552: temp/stck: 16r2D24A8 an OrderedCollection -16r1084/3551: temp/stck: 16r2D2490 an OwnedLock
and the int head frame... -16r108C OwnedLock(Process)>suspend 16r2D2490: a(n) OwnedLock -16r1084/3551: rcvr/clsr: 16r2D2490 an OwnedLock -16r1088/3550:cllr ip/ctxt: 16r81EFBB=8515515 -16r108C/3549: saved fp: -16r106C=-4204 -16r1090/3548: method: 16r10DF5B0 a CompiledMethod -16r1094/3547: flags: 16r1=1 numArgs: 0 hasContext: false isBlock: false -16r1098/3546: context: 16r41A280=4301440 -16r109C/3545: receiver: 16r2D2490 an OwnedLock -16r10A0/3544: temp/stck: 16r41A280 nil
What is the difference between ext and int head frames?
Err... That's an interpreter optimization. I am not sure I remember correctly. I think internal stack frame may have an incorrect pc / stack pointer / frame pointer as the value may be in the interpreter and not on stack. External stack frame have always the correct value. Basically if the frame is machine code frame, it is always external. If the frame is an interpreted frame, it is internal if it's the active stack frame. Or something like that.
Eliot can you explain again ? I'm pretty sure I am confused.
One thing I find curious here is "OwnedLock(Process)>suspend". Does this indicate that #suspend was sent to an OwnedLock and looked up the hierarchy to find the method in Process? But OwnedLock doesn't inherit from Process, it inherits from LinkedList.
Well then the external stack frame was correct and not the internal one.
I always look at machine code frame and they are always external, so I forgot :-(.
cheers -ben
On Fri, Jun 3, 2016 at 8:30 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jun 3, 2016 at 6:33 PM, Ben Coman btc@openinworld.com wrote:
where OwnedLock>>experiment1 is... | result | result := OrderedCollection new: 20. result myAdd: 0.
btw, OrderedCollection>>myAdd: is direct copy of OrderderedCollection>>add: just so [set break selector...] can distinguish these from every send of add: in the image.
The current VM produces a result of... OrderedCollection(0 11 12 21 22 13 14 24 8 9)
but I'm hoping to get something like... OrderedCollection(0 11 12 21 22 14 23 24 8 9)
After loading my 'ownedlock-reader.image' with an unchanged VM, then adding these two lines to transferTo:
objectMemory storeInteger: SuspendedPrimitiveFailCodeIndex ofObject: oldProc withValue: primFailCode. primFailCode := objectMemory fetchInteger: SuspendedPrimitiveFailCodeIndex ofObject: newProc.
then executing... OwnedLock new experiment1 ! I get an AssertionFailure in...
Spur32BitMMLESimulator>>fetchPointer: fieldIndex ofObject: objOop self assert: (self isForwarded: objOop) not. self assert: (fieldIndex >= 0 and: [fieldIndex < (self numSlotsOfAny: objOop) or: [fieldIndex = 0 "forwarders and free objs"]]). ^super fetchPointer: fieldIndex ofObject: objOop
where... objOop ==> 16r2D2490: a(n) OwnedLock 16r41A280 nil 16r41A280 nil 16r41A280 nil
fieldIndex ==> 3
self numSlotsOfAny: objOop ==> 3
One thing I'm curious about is why the " < " and not a " <= " ? super fetchPointer: fieldIndex ofObject: objOop successfully returns... 16r41A280: a(n) UndefinedObject
btw, here is the call stack... -16r106C [] in OwnedLock>experiment1 16r2D2490: a(n) OwnedLock -16r104C BlockClosure>on:do: 16r2D29C8: a(n) BlockClosure -16r1024 [] in OwnedLock>experiment1 16r2D2490: a(n) OwnedLock -16r1008 [] in BlockClosure>newProcess 16r2D27E8: a(n) BlockClosure
btw2, I am guessing that the lack of a Compiler in the call stack means we are within one of the forked blocks.
and the ext head frame... -16r106C [] in OwnedLock>experiment1 16r2D2490: a(n) OwnedLock -16r1064/3559: rcvr/clsr: 16r2D29C8 a BlockClosure -16r1068/3558:cllr ip/ctxt: 16rA3F922=10746146 -16r106C/3557: saved fp: -16r104C=-4172 -16r1070/3556: method: 16r81EF10 a CompiledMethod -16r1074/3555: flags: 16r1010001=16842753 numArgs: 0 hasContext: true isBlock: true -16r1078/3554: context: 16r2D29F8=2959864 -16r107C/3553: receiver: 16r2D2490 an OwnedLock -16r1080/3552: temp/stck: 16r2D24A8 an OrderedCollection -16r1084/3551: temp/stck: 16r2D2490 an OwnedLock
and the int head frame... -16r108C OwnedLock(Process)>suspend 16r2D2490: a(n) OwnedLock -16r1084/3551: rcvr/clsr: 16r2D2490 an OwnedLock -16r1088/3550:cllr ip/ctxt: 16r81EFBB=8515515 -16r108C/3549: saved fp: -16r106C=-4204 -16r1090/3548: method: 16r10DF5B0 a CompiledMethod -16r1094/3547: flags: 16r1=1 numArgs: 0 hasContext: false isBlock: false -16r1098/3546: context: 16r41A280=4301440 -16r109C/3545: receiver: 16r2D2490 an OwnedLock -16r10A0/3544: temp/stck: 16r41A280 nil
What is the difference between ext and int head frames?
One thing I find curious here is "OwnedLock(Process)>suspend". Does this indicate that #suspend was sent to an OwnedLock and looked up the hierarchy to find the method in Process? But OwnedLock doesn't inherit from Process, it inherits from LinkedList.
cheers -ben
TLDR; Reporting all the following helped me focus on the details, but most turned out irrelevant. I've left it for background info, but you may as well skip down to my Ahahhh.. moment. And the question I'm stuck on is right at the bottom an fairly isolated from everything else.
At the assertion failure, if I [print oop...] the Array inside the Ordered Collection it produces...
16r285008: a(n) Array 16r1 =0 (16r0) 16r17 =11 (16rB) 16r19 =12 (16rC) 16r2B =21 (16r15) 16r2D =22 (16r16) 16r1D =14 (16rE) 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil
which is somewhat positive, showing that now experimentSuccessAndSleep did not error(13) i.e. OrderedCollection(0 11 12 21 22 14)
Also, it seems it got to the end of the first forked block, and considering the later bytecode trace, maybe the problem is with terminating that forked block. Is line "145 7D returnTopFromBlock (2506913)" below where this starts to happen?
With [break on selector...] > myAdd: I can <proceed> successfully five times. Turning on [print sends] and [print bytecode each bytecode] then proceding the sixth time produces...
2506895 myAdd: 21 70 pushReceiverBytecode (2506897) 22 10 pushTemporaryVariableBytecode (2506898) 23 E0 sendLiteralSelector1ArgBytecode (2506899)
OrderedCollection>addLast: 2506898 addLast: 25 00 pushReceiverVariableBytecode (2506900) 26 C2 bytecodePrimSize (2506901) 27 02 pushReceiverVariableBytecode (2506902) 28 B6 bytecodePrimEqual (2506903) 33 00 pushReceiverVariableBytecode (2506904) 34 02 pushReceiverVariableBytecode (2506905) 35 76 pushConstantOneBytecode (2506906) 36 B0 bytecodePrimAdd (2506907) 37 81 extendedStoreBytecode (2506908) 39 10 pushTemporaryVariableBytecode (2506909) 40 C1 bytecodePrimAtPut (2506910) 41 7C returnTopFromMethod (2506911) 24 7C returnTopFromMethod (2506912) 145 7D returnTopFromBlock (2506913) "localReturnValue := self internalStackTop." "self commonCallerReturn" " self setMethod: (self frameMethod: localFP)." " self fetchNextBytecode" " self internalStackTopPut: localReturnValue (=14)" "I am unable to [print call stack] at this point ==> invalid frame pointer Is this normal at this point?"
55 87 popStackBytecode (2506914) "Still unable to [print call stack] at this point ==> invalid frame pointer 56 45 pushLiteralVariableBytecode (2506915) 57 D4 sendLiteralSelector0ArgsBytecode (2506916) "rcvr = ProcessorScheduler:"
ProcessorScheduler>activeProcess 2506915 activeProcess 21 01 pushReceiverVariableBytecode (2506917) --> Process 22 D0 sendLiteralSelector0ArgsBytecode (2506918) --> effectiveProcess
Process>effectiveProcess 2506917 effectiveProcess 21 05 pushReceiverVariableBytecode (2506919) 22 88 duplicateTopBytecode (2506920) 23 73 pushConstantNilBytecode (2506921) 24 C6 bytecodePrimIdentical (2506922) 26 87 popStackBytecode (2506923) 27 70 pushReceiverBytecode (2506924) 28 7C returnTopFromMethod (2506925) 23 7C returnTopFromMethod (2506926) 58 D3 sendLiteralSelector0ArgsBytecode (2506927) --> #suspend "messageSelector = #suspend" "rcvr = 16r24B748: a(n) Process 16r41A280 nil 16r41A280 nil 16r91 =72 (16r48) "priority" 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r1 =0 (16r0) "suspendedPrimitiveFailCode, newly introduced"
Process>suspend 2506926 suspend "commonSendOrdinary" " internalExecuteNewMethod" " slowPrimitiveResponse" " dispatchFunctionPointer: " " primitiveSuspend" " process := self stackTop. " process = self activeProcess ifTrue: " [self pop: 1 thenPush: objectMemory nilObject. " ^self transferTo: self wakeHighestPriority].
In transferTo:, newProc is.... 16r24B9B8: a(n) Process 16r41A280 nil 16r24BAD8 a MethodContext 16r8F =71 (16r47) 16r459620 a LinkedList 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r41A280 nil 16r55 =42 (16r2A)
Ahahhhh.... priority 71, that is the following forked process... [result myAdd: 21. [result myAdd: 22. self experimentFailAndWakeOther] on: Error do: [result myAdd: 23]. result myAdd: 24] forkAt: 71.
and its suspendedPrimitiveFailCode ==> 42 as set by primitiveExperimentFailAndWakeOther And indeed, tracing through transferTo:, primFailCode := 42.
So in the simulated image had executed experimentFailAndWakeOther at priority 71 which woke up experimentSuccessAndSleep at priority 72, transfering execution to "myAdd: 14", and when that block finished, execution transferred back experimentFailAndWakeOther with its saved primFailCode.
Now #slowPrimitiveResponse returned to internalExecuteNewMethod with succeeded := false "correct per experiment" and dropped through #internalExecuteNewMethod to #internalActivateNewMethod. which does... methodHeader := objectMemory methodHeaderOf: newMethod. where newMethod is... 16r10DF5B0: a(n) CompiledMethod nbytes 48 16rA0009 =327684 (16r50004) 16r5B0278 #remove:ifAbsent: 16r5C8700 #ifNil: 16r14B5030 an AdditionalMethodState 16r9D2A78 a ClassBinding #Process -> 16r0183C860 and not the expected #experimentFailAndWakeOther
So /newMethod/ also needed to be saved across the context switch. In the simulated image I added another ivar for this to Process for this, and these lines into #transferTo:
objectMemory storePointer: SuspendedNewMethodIndex ofObject: oldProc withValue: newMethod. newMethod := objectMemory fetchPointer: SuspendedNewMethodIndex ofObject: newProc.
Now an assert failed in #checkForAndFollowForwardedPrimitiveState self assert: (self saneFunctionPointerForFailureOfPrimIndex: primIndex)].
but we have correctly restored newMethod = 16rC0DA18: a(n) CompiledMethod nbytes 34 16r20009 =65540 (16r10004) 16r1790AB0 #iAmMethodExperimentFailAndWakeOther 16r5C6630 #primitiveFail 16r147A3B8 an AdditionalMethodState 16r17D1090 a ClassBinding #OwnedLock -> 16r01116370
and correctly restored primIndex = 561. but #saneFunctionPointerForFailureOfPrimIndex: uses a a stale primitiveFunctionPointer which needs to be saved across the context switch.
--------------- Now I am a little confused about primitiveFunctionPointer. By name I would expect a pointer and to use #storePointer:toObject:withValue , but actually I find... primitiveFunctionPointer ==> #primitiveSuspend (which btw is stale) self functionPointerFor: primIndex inClass: objectMemory nilObject ==> #primitiveExperimentFailAndWakeOther (which is correct)
So what store method should I use for primitiveFunctionPointer ?
cheers -ben
On Sat, Jun 4, 2016 at 10:10 AM, Ben Coman btc@openinworld.com wrote:
Now an assert failed in #checkForAndFollowForwardedPrimitiveState self assert: (self saneFunctionPointerForFailureOfPrimIndex: primIndex)].
but we have correctly restored newMethod = 16rC0DA18: a(n) CompiledMethod nbytes 34 16r20009 =65540 (16r10004) 16r1790AB0 #iAmMethodExperimentFailAndWakeOther 16r5C6630 #primitiveFail 16r147A3B8 an AdditionalMethodState 16r17D1090 a ClassBinding #OwnedLock -> 16r01116370
and correctly restored primIndex = 561. but #saneFunctionPointerForFailureOfPrimIndex: uses a a stale primitiveFunctionPointer which needs to be saved across the context switch.
Now I am a little confused about primitiveFunctionPointer. By name I would expect a pointer and to use #storePointer:toObject:withValue , but actually I find... primitiveFunctionPointer ==> #primitiveSuspend (which btw is stale) self functionPointerFor: primIndex inClass: objectMemory nilObject ==> #primitiveExperimentFailAndWakeOther (which is correct)
So what store method should I use for primitiveFunctionPointer ?
cheers -ben
As a quick check, under the assumption that a newly opened image doesn't have any forwarders for a primitive to fail on, and that the "reader" image probably hasn't done any becomes, I just commented the call to #maybeRetryFailureDueToForwarding out of #slowPrimitiveResponse. Thus running...
OwnedLock new experiment1. !
produces...
OrderedCollection(0 11 12 21 22 14 23 24 8 9)
which is exactly what I was hoping for.
So having *mostly* produced a technical proof of concept (for StackVM), the meta-question is whether a change like this worth pursuing? Apart from my own need for the OwnedLock primitiveWaitAcquire, I have cluelessly wondered if there might be any benefit for multi-threading or callback patterns, now or in the future.
Also, are there any obvious/known difficulties to be expected doing this with JIT side?
cheers -ben
P.S. I could never imagine getting so far in a few days without the Simulator. It is **really** cool to be bale to load an Image on an unmodified VM, then switch in some new VM code while the Image is running.
vm-dev@lists.squeakfoundation.org