<div dir="ltr">Hi Eliot, Thanks for the detailed response.<br><br><div class="gmail_quote"><div dir="ltr">On Tue, 28 Aug 2018 at 04:21, Eliot Miranda <<a href="mailto:eliot.miranda@gmail.com">eliot.miranda@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_quote"><div dir="ltr">On Mon, Aug 27, 2018 at 11:36 AM Ben Coman <<a href="mailto:btc@openinworld.com" target="_blank">btc@openinworld.com</a>> wrote:</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>Back when I was having a go at new mutex primitives, </div><div>when a process "A" failed to lock a mutex, I wanted to return </div><div>a primitive-failed-code in addition to the usual context-change to different process "B"</div><div>However what I observed was that because of the context change </div><div>the primitive-failed-code incorrectly ended up returned on the stack of process "B".</div></div></div></blockquote><div><br></div><div>Your description doesn't match how (I understand) the VM works. The only way that Process A can initiate a process switch, mutex lock, et al, is by sending a message to some object (a process, mutex or semaphore). So we're talking about Process>>suspend & resume as well as the lock/unlock and wait/signal primitives. Primitive failure *always* delivers a primitive error to the method that contains the primitive and, in this case, initiated the process switch. Primitives validate their arguments and then succeed, or fail, leaving their arguments undisturbed (there is one regrettable exception to this in the BttF/Cog VM which is the segment loading primitive that leaves its input word array scrambled if a failure occurs, rather than incur the cost of cloning the array).</div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div><br></div><div>So the only way that a primitive could fail and the error code end up on the wrong process's stack would be if the primitive was mis-designed to not validate before occurring. </div></div></div></blockquote><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>Essentially it can not fail and cause a side effect. </div></div></div></blockquote><div><br></div><div>This was my first foray into writing a primitive (still on my backlog to be completed).</div><div>I was aware of validating the arguments and leaving them undisturbed for a failure, </div><div>but wasn't paying attention to primitive failure being completely free from side effects. </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>Primitives should be side-effect free when they fail and hence if a process switch primitive fails, it cannot yet have caused a process switch and therefore the error code would have to be delivered to process A's stack.</div></div></div></blockquote><div><br></div><div><div>That is kind of the outcome of what I was proposing. I was trying for a mutex locking primitive that could fail without causing a side-effect "in the image". Maybe its not valid to distinguish between side-effects "inside" or "outside" the image, but I thought it might be reasonable for a flag hidden "in the VM" to be just another event checked by #checkForEventsMayContextSwitch: . Effectively the "in image" effect happens outside the primitive, a bit like reaching nextWakeupUsecs, or like I imagine a callback might work.</div><div><br></div><div>A side thought was that if context-changes occurred in a *single* location in #checkForEventsMayContextSwitch, </div><div>it might be easier to make an "Idle VM"</div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div><br></div><div>I'm stretching my memory so there is a reasonable change this is misleading...</div><div>but I believe I observed this happening in CoInterpreter>>internalExecuteNewMethod </div><div>near this code..</div><div><br></div><div> "slowPrimitiveResponse may of course context-switch. ..."</div><div> succeeded := self slowPrimitiveResponse.</div><div> ...</div><div> succeeded ifTrue: [....</div></div></div></blockquote><div><br></div><div>But internalExecuteNewMethod doesn't contain the switch code, internalActivateNewMethod does, and it does the process switch *after* delivering the primitive failure code, see reapAndResetErrorCodeTo:header: in the following:</div><div><br></div><div>internalActivateNewMethod<br></div><div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>...</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>(self methodHeaderHasPrimitive: methodHeader) ifTrue:</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>["Skip the CallPrimitive bytecode, if it's there, and store the error code if the method starts</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span> with a long store temp. Strictly no need to skip the store because it's effectively a noop."</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span> localIP := localIP + (self sizeOfCallPrimitiveBytecode: methodHeader).</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span> primFailCode ~= 0 ifTrue:</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>[self reapAndResetErrorCodeTo: localSP header: methodHeader]].</div><div><br></div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>self assert: (self frameNumArgs: localFP) == argumentCount.</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>self assert: (self frameIsBlockActivation: localFP) not.</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>self assert: (self frameHasContext: localFP) not.</div><div><br></div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>"Now check for stack overflow or an event (interrupt, must scavenge, etc)."</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>localSP < stackLimit ifTrue:</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>[self externalizeIPandSP.</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span> switched := self handleStackOverflowOrEventAllowContextSwitch:</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span>(self canContextSwitchIfActivating: newMethod header: methodHeader).</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span> self returnToExecutive: true postContextSwitch: switched.</div><div><span class="gmail-m_8629604044109620616gmail-Apple-tab-span" style="white-space:pre-wrap"> </span> self internalizeIPandSP]</div></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div><br></div><div>Though I can't exactly put my finger on explaining why, my intuition is that </div><div>changing threads "half way" through a bytecode is a bad thing. </div></div></div></blockquote><div><br></div><div>Indeed it is, and the VM does not do this. It is possible that the execution simulation machinery in Context, InstructionStream at al could have been written carelessly to allow this to occur, but it cannot and does not occur in the VM proper.</div></div></div></blockquote><div><br></div><div><div> I made a chart to understand this better. One thing first, I'm not sure I've correctly linked execution of the primitives into slowPrimitiveResponse. I'm not at all clear about how internalExecuteNewMethod selects between internalQuickPrimitiveResponse </div><div>and slowPrimitiveResponse, and what is the difference between them?</div></div><div><br></div><div><div><img src="cid:ii_jle2plu34" alt="ContextChange-Existing.png" width="472" height="260"><br></div></div><div><br></div><div><div>So I understand that checkForEventsMayContextSwitch: called at the end of internalActivateNewMethod<br>occurs after bytecode execution has completed, so that context switches made there are done "between" bytecodes.</div><div>However my perspective is that internalExecuteNewMethod is only half way through a bytecode execution when </div><div>the primitives effect context changes. So internalActivateNewMethod ends up working on a different Process than internalExecuteNewMethod started with. The bottom three red lines in the chart are what I considered to be changing threads "half way" through a bytecode. </div><div><br></div><div>Ahh, I'm slowly coming to grips with this. It was extremely confusing at the time why my failure code from the primitive was turning up in a different Process, though I then learnt a lot digging to discover why. In summary, if the primitive succeeds it simply returns from internalExecuteNewMethod and internalActivateNewMethod never sees the new Process B. My problem violating the primitive-failure side-effect rule was that internalActivateNewMethod trying to run Process A in-Image-primitive-failure code </div><div>instead ran Process B in-Image-primitive-failure code.</div></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>Interestingly the comment in #transferTo:from says... <br></div><div> "Record a process to be awoken on the next interpreter cycle."</div><div>which sounds like what I'd propose, but actually it doesn't wait for </div><div>the next interpreter cycle and instead immediately changes context.</div></div></div></blockquote><div><br></div><div>No it doesn't. It effects the process change, but control continues with the caller, allowing the caller to do other things before the process resumes. </div></div></div></blockquote><div><br></div><div>In my case I believe "control continues with the caller" was not true since internalActivateNewMethod </div><div>was trying to run the in-Image-primitive-failure code after the process changed.</div><div>But that was because I violated the side-effect rule.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>For example, if in checkForEventsMayContextSwitch: more than one semaphore is signaled (it could initiate any combination of signals of the low space semaphore, the input semaphore, external semaphores (associated with file descriptors & sockets), and the delay semaphore) then only the highest priority process would be runnable after the sequence, and several transferTo:[from:]'s could have been initiated from these signals, depending on process priority. But checkForEventsMayContextSwitch: will not finish mid-sequence. It will always complete all of its signals before returning to code that can then resume the newly activated process.</div></div></div></blockquote><div><br></div><div>Just to summarise to check I understood this correctly, no bytecode is executed during checkForEventsMayContextSwitch:. </div><div>That is, its multiple transferTo: calls don't re-enter the interpreter? It just which Process is set to run changes,</div><div>until at the end of checkForEventsMayContextSwitch it returns to the interpreter to pick up the next bytecode of the active process.</div><div><br></div><div>cheers -ben. </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
</blockquote></div></div>