<div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Andrei,</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 11, 2020 at 11:48 AM Andrei Chis <<a href="mailto:chisvasileandrei@gmail.com">chisvasileandrei@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> <div dir="ltr"><div dir="ltr">Hi Eliot,<div><br></div><div>Thanks for the answer. That helps to understand what is going on and it can explain why just adding a call to `self pc` makes the crash disappear. </div><div><br></div><div>Just what was maybe not obvious in my previous email is that we get this problem more or less randomly. We have tests for verifying that tools work when various extensions raise exceptions (these tests copy the stack). Sometimes they work correctly and sometimes they crash. These crashes happen in various tests and until now the only common thing we noticed is that the pc of the contexts where the crash happens looks off. Also the contexts in which this happens are at the beginning of the stack so part of a long computation (it gets copied multiple times).</div><div><br></div><div>Initially we suspected that there is some memory corruption somewhere due to external calls/memory. Just the fact that calling `self pc` before seems to fix the issue reduces those chances. But who knows.</div></div></div></blockquote><div><br></div><div>Well, it does look like a VM bug. The VM is somehow failing to intercept some access, perhaps in shallow copy. Weird. I shall try and reproduce. Is there anything special about the process you copy using copyTo: ?</div><div><br></div><div>(see below)</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>On Fri, Sep 11, 2020 at 6:36 PM Eliot Miranda <<a href="mailto:eliot.miranda@gmail.com" target="_blank" style="font-size:11px">eliot.miranda@gmail.com</a>> wrote:<br></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> <div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Andrei,<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 11, 2020 at 8:58 AM Andrei Chis <<a href="mailto:chisvasileandrei@gmail.com" target="_blank">chisvasileandrei@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> <div dir="ltr">Hi,<div><br></div><div>We are getting often crashes on our CI when calling `Context>copyTo:` in a GT image and a vm build from <a href="https://github.com/feenkcom/opensmalltalk-vm" target="_blank">https://github.com/feenkcom/opensmalltalk-vm</a>.</div><div><br></div><div>To sum up during `Context>copyTo:`, `Object>>#copy` is called on a context leading to a segmentation fault crash. Looking at that context in lldb the pc looks off. It has the value `0xfffffffffea7f6e1`.</div><div><div><br></div><div><pre style="box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:11.9px;margin-top:0px;margin-bottom:16px;padding:16px;overflow:auto;line-height:1.45;background-color:rgb(246,248,250);border-top-left-radius:6px;border-top-right-radius:6px;border-bottom-right-radius:6px;border-bottom-left-radius:6px;color:rgb(36,41,46)"><code style="box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:11.9px;padding:0px;margin:0px;border-top-left-radius:6px;border-top-right-radius:6px;border-bottom-right-radius:6px;border-bottom-left-radius:6px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit"> (lldb) call (void *) printOop(0x1206b6990)
0x1206b6990: a(n) Context
0x1206b6a48 0xfffffffffea7f6e1 0x9 0x1146b2e08 0x1206b6b00
0x1206b6b28 0x1206b6b50 <br></code></pre></div></div><div><br></div><div>Can this indicate some corruption or is it expected to have such values? `CoInterpreter>>ensureContextHasBytecodePC:` has code that also handles negative values for the pc which suggests that this might be expected.<br></div></div></blockquote><div><br></div><div>The issue is that that value is expected *inside* the VM. It is the frame pointer for the context. But above the Vm this value should be hidden. The VM should intercept all accesses to such fields in contexts and automatically map them back to the appropriate values that the image expects to see. [The same thing is true for CompiledMethods; inside the VM methods may refer to their JITted code, but this is invisible from the image]. Intercepting access to Context state already happens with inst var access in methods, with the shallowCopy primitive, with instVarAt: et al, etc.</div><div><br></div><div>So I expect the issue here is that copyTo: invokes some primitive which does not (yet) check for a context receiver and/or argument, and hence accidentally it reveals the hidden state to the image and a crash results. What I need to know are the definitions for copyTo: and copy, etc all the way down to primitives.</div></div></div></div></div></blockquote><div><br></div><div>Here is the source code:</div></div></div></blockquote><div><br></div><div>Cool, nothing unusual here. This should all work perfectly. Tis a VM bug. However...</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>Context >> copyTo: aContext <br></div><div> "Copy self and my sender chain down to, but not including, aContext. End of copied chain will have nil sender."<br> | copy |<br> self == aContext ifTrue: [^ nil].<br> copy := self copy.<br> self sender ifNotNil: [<br> copy privSender: (self sender copyTo: aContext)].<br> ^ copy<br></div></div></div></blockquote><div><br></div><div>Let me suggest</div><div><br></div><span style="color:rgb(0,0,0)">Context >> copyTo: aContext </span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)"> "Copy self and my sender chain down to, but not including, aContext. End of copied chain will have nil sender."</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)"> | copy |</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)"> self == aContext ifTrue: [^ nil].</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)"> copy := self copy.</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)"> self sender ifNotNil:</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)"> [:mySender| copy privSender: (mySender copyTo: aContext)].</span><br style="color:rgb(0,0,0)"><div><span style="color:rgb(0,0,0)"> ^ copy</span> </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div></div><div>Object>>#copy<br></div><div> ^self shallowCopy postCopy<br><br></div>Object >> shallowCopy<br> | class newObject index |<br> <primitive: 148><br> class := self class.<br> class isVariable<br> ifTrue: <br> [index := self basicSize.<br> newObject := class basicNew: index.<br> [index > 0]<br> whileTrue: <br> [newObject basicAt: index put: (self basicAt: index).<br> index := index - 1]]<br> ifFalse: [newObject := class basicNew].<br> index := class instSize.<br> [index > 0]<br> whileTrue: <br> [newObject instVarAt: index put: (self instVarAt: index).<br> index := index - 1].<br> ^ newObject<br><div><br></div><div>The code of the primitiveClone looks the same [1]</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_quote"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>Changing `Context>copyTo:` by adding a `self pc` before calling `self copy` leads to no more crashes. Not sure if there is a reason for that or just plain luck.<br></div><div><br></div><div>A simple reduced stack is below (more details in this issue [1]). The crash happens always with contexts reified as objects (in this case 0x1206b6990 s [] in GtExamplesCommandLineHandler>runPackages). </div><div>Could this suggest some kind of issue in the vm when reifying contexts, or just some other problem with memory corruption?</div></div></blockquote><div><br></div><div>This looks like an oversight in some primitive. Here for example is the implementation of the shallowCopy primitive, a.k.a. clone, and you can see where it explcitly intercepts access to a context.</div><div><br></div><div><div>primitiveClone</div><div><span style="white-space:pre-wrap"> </span>"Return a shallow copy of the receiver.</div><div><span style="white-space:pre-wrap"> </span> Special-case non-single contexts (because of context-to-stack mapping).</div><div><span style="white-space:pre-wrap"> </span> Can't fail for contexts cuz of image context instantiation code (sigh)."</div><div><br></div><div><span style="white-space:pre-wrap"> </span>| rcvr newCopy |</div><div><span style="white-space:pre-wrap"> </span>rcvr := self stackTop.</div><div><span style="white-space:pre-wrap"> </span>(objectMemory isImmediate: rcvr)</div><div><span style="white-space:pre-wrap"> </span>ifTrue:</div><div><span style="white-space:pre-wrap"> </span>[newCopy := rcvr]</div><div><span style="white-space:pre-wrap"> </span>ifFalse:</div><div><span style="white-space:pre-wrap"> </span>[(objectMemory isContextNonImm: rcvr)</div><div><span style="white-space:pre-wrap"> </span>ifTrue:</div><div><span style="white-space:pre-wrap"> </span>[newCopy := self cloneContext: rcvr]</div><div><span style="white-space:pre-wrap"> </span>ifFalse:</div><div><span style="white-space:pre-wrap"> </span>[(argumentCount = 0</div><div><span style="white-space:pre-wrap"> </span> or: [(objectMemory isForwarded: rcvr) not])</div><div><span style="white-space:pre-wrap"> </span>ifTrue: [newCopy := objectMemory clone: rcvr]</div><div><span style="white-space:pre-wrap"> </span>ifFalse: [newCopy := 0]].</div><div><span style="white-space:pre-wrap"> </span>newCopy = 0 ifTrue:</div><div><span style="white-space:pre-wrap"> </span>[^self primitiveFailFor: PrimErrNoMemory]].</div><div><span style="white-space:pre-wrap"> </span>self pop: argumentCount + 1 thenPush: newCopy</div></div><div><br></div><div>But since Squeak doesn't have copyTo: I have no idea what primitive is being used. I'm guessing 168 primitiveCopyObject, which seems to check for a Context receiver, but not for a CompiledCode receiver. What does the primitive failure code look like? Can you post the copyTo: implementations here please?</div></div></div></div></div></blockquote><div><br></div><div>The code is above. I also see Context>>#copyTo: in Squeak calling also Object>>copy for contexts.</div><div><br></div><div><div dir="ltr"><div>When a crash happens we don't get the exact same error all the time. For example we get most often on mac:</div><div><br></div><div>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Process 35690 stopped</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">* thread #1, queue = </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(47,180,29)">'com.apple.main-thread'</span><span style="font-variant-ligatures:no-common-ligatures">, stop reason = </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(180,36,25)">EXC_BAD_ACCESS (code=EXC_I386_GPFLT)</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(159,160,28)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"><span> </span>frame #0: </span><span style="font-variant-ligatures:no-common-ligatures">0x00000001100b1004</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">-><span> </span>0x1100b1004: inl<span> </span>$0x4c, %eax</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>0x1100b1006: leal <span> </span>-0x5c(%rip), %eax</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>0x1100b100c: pushq<span> </span>%r8</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>0x1100b100e: movabsq $0x1109e78e0, %r9 <span> </span>; imm = 0x1109E78E0<span> </span></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Target 0: (GlamorousToolkit) stopped.</span></p></div><div><br></div></div><div><br></div><div>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Process 29929 stopped</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(180,36,25)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">* thread #1, queue = </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(47,180,29)">'com.apple.main-thread'</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">, stop reason = </span><span style="font-variant-ligatures:no-common-ligatures">EXC_BREAKPOINT (code=EXC_I386_BPT, subcode=0x0)</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(159,160,28)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"><span> </span>frame #0: </span><span style="font-variant-ligatures:no-common-ligatures">0x00000001100fe7ed</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">-><span> </span>0x1100fe7ed: int3<span> </span></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>0x1100fe7ee: int3<span> </span></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>0x1100fe7ef: int3<span> </span></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>0x1100fe7f0: int3<span> </span></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Target 0: (GlamorousToolkit) stopped.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"></p></div></div><div><br></div><div>[1] <a href="https://github.com/feenkcom/opensmalltalk-vm/blob/5f7d49227c9599a35fcb93892b727c93a573482c/smalltalksrc/VMMaker/StackInterpreterPrimitives.class.st#L325" target="_blank">https://github.com/feenkcom/opensmalltalk-vm/blob/5f7d49227c9599a35fcb93892b727c93a573482c/smalltalksrc/VMMaker/StackInterpreterPrimitives.class.st#L325</a><br></div><div><br></div><div>Cheers,</div><div>Andrei</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_quote"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><pre style="box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:11.9px;margin-top:0px;margin-bottom:16px;padding:16px;overflow:auto;line-height:1.45;background-color:rgb(246,248,250);border-top-left-radius:6px;border-top-right-radius:6px;border-bottom-right-radius:6px;border-bottom-left-radius:6px;color:rgb(36,41,46)"><code style="box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:11.9px;padding:0px;margin:0px;border-top-left-radius:6px;border-top-right-radius:6px;border-bottom-right-radius:6px;border-bottom-left-radius:6px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit"> 0x7ffeefbb4380 M Context(Object)>copy 0x1206b6990: a(n) Context
0x7ffeefbb43b8 M Context>copyTo: 0x1206b6990: a(n) Context
0x7ffeefbb4400 M Context>copyTo: 0x1206b5ae0: a(n) Context
...
0x7ffeefba6078 M Context>copyTo: 0x110548b28: a(n) Context
0x7ffeefba60d0 I Context>copyTo: 0x110548a70: a(n) Context
0x7ffeefba6118 I MessageNotUnderstood(Exception)>freezeUpTo: 0x110548a20: a(n) MessageNotUnderstood
0x7ffeefba6160 I MessageNotUnderstood(Exception)>freeze 0x110548a20: a(n) MessageNotUnderstood
0x7ffeefba6190 M [] in GtExampleEvaluator>result 0x110544fb8: a(n) GtExampleEvaluator
0x7ffeefba61c8 M BlockClosure>cull: 0x110545188: a(n) BlockClosure
0x7ffeefba6208 M Context>evaluateSignal: 0x110548c98: a(n) Context
0x7ffeefba6240 M Context>handleSignal: 0x110548c98: a(n) Context
0x7ffeefba6278 M Context>handleSignal: 0x110548be0: a(n) Context
0x7ffeefba62b0 M MessageNotUnderstood(Exception)>signal 0x110548a20: a(n) MessageNotUnderstood
0x7ffeefba62f0 M GtDummyExamplesWithInheritanceSubclassB(Object)>doesNotUnderstand: exampleH 0x1105487d8: a(n) GtDummyExamplesWithInheritanceSubclassB
0x7ffeefba6328 M GtExampleEvaluator>primitiveProcessExample:withEvaluationContext: 0x110544fb8: a(n) GtExampleEvaluator
...
0x7ffeefbe64d0 M [] in GtExamplesHDReport class(HDReport class)>runPackages: 0x1145e41c8: a(n) GtExamplesHDReport class
0x7ffeefbe6520 M [] in Set>collect: 0x1206b5ab0: a(n) Set
0x7ffeefbe6568 M Array(SequenceableCollection)>do: 0x1206b5c50: a(n) Array
0x1206b5b98 s Set>collect:
0x1206b5ae0 s GtExamplesHDReport class(HDReport class)>runPackages:
0x1206b6990 s [] in GtExamplesCommandLineHandler>runPackages
0x1206b6a48 s BlockClosure>ensure:
0x1206b6b68 s UIManager class>nonInteractiveDuring:
0x1206b6c48 s GtExamplesCommandLineHandler>runPackages
0x1206b6d98 s GtExamplesCommandLineHandler>activate
0x1206b75d0 s GtExamplesCommandLineHandler class(CommandLineHandler class)>activateWith:
0x1207d2f00 s [] in PharoCommandLineHandler(BasicCommandLineHandler)>activateSubCommand:
0x1207e6620 s BlockClosure>on:do:
0x1207f7ab8 s PharoCommandLineHandler(BasicCommandLineHandler)>activateSubCommand:
0x120809d40 s PharoCommandLineHandler(BasicCommandLineHandler)>handleSubcommand
0x12082ca60 s PharoCommandLineHandler(BasicCommandLineHandler)>handleArgument:
0x120789938 s [] in PharoCommandLineHandler(BasicCommandLineHandler)>activate
0x1207a83e0 s BlockClosure>on:do:
0x1207b57a0 s [] in PharoCommandLineHandler(BasicCommandLineHandler)>activate
0x1207bf830 s [] in BlockClosure>newProcess</code></pre></div><div>Cheers,</div><div>Andrei</div><div><br></div><div><br></div><div>[1] <a href="https://github.com/feenkcom/gtoolkit/issues/1440" target="_blank">https://github.com/feenkcom/gtoolkit/issues/1440</a></div><div><br></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div></div></div></div>
</blockquote></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div></div></div>