<br><br><div class="gmail_quote">On Mon, Jul 19, 2010 at 1:10 PM, Schwab,Wilhelm K <span dir="ltr"><<a href="mailto:bschwab@anest.ufl.edu">bschwab@anest.ufl.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
If the FFI call is flawed, why does it work when not stepping in the debugger? I'm not buying it :)<br></blockquote><div><br></div><div>A few people have seen problems with the part of the debugger in question. Here's the canonical version of the method by Andreas 'ar 5/25/2000 20:41'. It is invoked from ContextPart>>doPrimitive:method:receiver:args: and its job is to invoke just the primitive and either answer the result or a special PrimitiveFailed token. The debugger then checks for the result being the PrimitiveFailed token and if so steps into the method.</div>
<div><br></div><div><div>ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs: arguments</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"Hack. Attempt to execute the named primitive from the given compiled method"</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>| selector theMethod spec |</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>arguments size > 8 ifTrue:[^PrimitiveFailToken].</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>selector _ #(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive: </div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with: </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with: </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:with:</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:with:with:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:with:with:with:) at: arguments size+1.</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>theMethod _ aReceiver class lookupSelector: selector.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>theMethod == nil ifTrue:[^PrimitiveFailToken].</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>spec _ theMethod literalAt: 1.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt: 1) startingAt: 1.</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>^aReceiver perform: selector withArguments: arguments</div></div><div> </div><div>What this does is slam the named primitive to be evaluated into the relevant implementation of tryNamedPrimitive[:with:with:with:with:with:with:with:] and invoke it. One minor problem is that there are only 9 methods so there's only support for 0 to 8 arguments. The real problem with this is that it doesn't flush the VM's method lookup cache and so one can end up invoking a different named primitive, one that was installed into the same method by a previous invocation of tryNamedPrimitiveIn:for:withArgs:.</div>
<div><br></div><div>Michael Haupt saw the effect of this doing bytecode simulation on Cog. For some reason it's much more likely to hit the cache flush issue in Cog tan in the standard interpreter (probably because Cog makes less use of the method cache because machine code sends have their send caches inline in the machine code).</div>
<div><br></div><div>I fixed this for Michael by using the following version 'eem 5/23/2010 14:13'</div><div>ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs: arguments</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"Hack. Attempt to execute the named primitive from the given compiled method"</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>| selector theMethod spec |</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>arguments size > 8 ifTrue:[^PrimitiveFailToken].</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>selector := #(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive: </div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with: </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with: </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:with:</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:with:with:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:with:with:with:) at: arguments size+1.</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>theMethod := aReceiver class lookupSelector: selector.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>theMethod == nil ifTrue:[^PrimitiveFailToken].</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>spec := theMethod literalAt: 1.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt: 1) startingAt: 1.</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>theMethod flushCache.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>selector flushCache.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>^aReceiver perform: selector withArguments: arguments</div>
<div><br></div><div>It adds "theMethod flushCache. selector flushCache" to flush the cache and at least for a couple of months things seemed to be OK. But Michael sent me a different case a week ago that turns out to be due to exactly the same cause. When simulated his code was trying to invoke the primDigitDivNegative primitive via digitDiv:neg: (which doesn't exist in Cog and so should simply fail) but it was succeeding, running some bogus primitive.</div>
<div><br></div><div>There is a more fundamental issue with the above. try debugging it. If you do you stand a good chance of the debugger overwriting what the debugger version of the method is trying to effect. That's why if you look at the analogous machinery for num bered primitives you'll see the above approach (which was used in Smalltalk-80 V2) is discarded and a much more reliable approach, a primitive-executing primitive, is used, namely ProtoObject>>#tryPrimitive:withArgs: 'ajh 1/31/2003 22:21':</div>
<div>ProtoObject>>tryPrimitive: primIndex withArgs: argumentArray</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"This method is a template that the Smalltalk simulator uses to </div><div>
<span class="Apple-tab-span" style="white-space:pre"> </span>execute primitives. See Object documentation whatIsAPrimitive."</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span><primitive: 118></div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>^ ContextPart primitiveFailToken</div><div><br></div><div>In the Teleplace images I've taken a similar approach, providing a primitive in Cog. The below is slightly different because the Teleplace images debugger supports primitive error codes (and I need to fold this back into Squeak asap). ContextPart>>tryNamedPrimitiveIn:for:withArgs: eem 5/12/2009</div>
<div><br></div><div><div>ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs: arguments</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>| selector theMethod spec receiverClass |</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span><primitive: 218 error: ec></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ec ifNotNil:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>["If ec is an integer other than -1 there was a problem with primitive 218,</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span> not with the external primitive itself. -1 indicates a generic failure (where</div><div><span class="Apple-tab-span" style="white-space:pre"> </span> ec should be nil) but ec = nil means primitive 218 is not implemented. So</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span> interpret -1 to mean the external primitive failed with a nil error code."</div><div><span class="Apple-tab-span" style="white-space:pre"> </span> ec isInteger ifTrue:</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>[ec = -1</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ifTrue: [ec := nil]</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ifFalse: [self primitiveFailed]].</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>^{PrimitiveFailToken. ec}].</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"Assume a nil error code implies the primitive is not implemented and fall back on the old code."</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>"Hack. Attempt to execute the named primitive from the given compiled method"</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>arguments size > 8 ifTrue:</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>[^{PrimitiveFailToken. nil}].</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>selector := #(</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive </div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive: </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with: </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with: </div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:</div><div>
<span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:with:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:with:with:</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>tryNamedPrimitive:with:with:with:with:with:with:with:) at: arguments size+1.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>receiverClass := self objectClass: aReceiver.</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>theMethod := receiverClass lookupSelector: selector.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>theMethod == nil ifTrue:</div><div>
<span class="Apple-tab-span" style="white-space:pre"> </span>[^{PrimitiveFailToken. nil}].</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>spec := theMethod literalAt: 1.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt: 1) startingAt: 1.</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>Smalltalk unbindExternalPrimitives.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>^self object: aReceiver perform: selector withArguments: arguments inClass: receiverClass</div>
</div><div><br></div><div>With this approach there's no argument count limit and there's no chance of executing the wrong primitive. But it requires a primitive in the VM. Perhaps the VM folks can take a look at porting the Cog primitive primitiveDoNamedPrimitiveWithArgs back to the base VM and then we can use something like the above with the fallback code, liberally sprinkled with cache flushes, only as a fallback.</div>
<div><br></div><div>HTH</div><div>Eliot</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div><div></div><div class="h5"><br>
<br>
<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:pharo-project-bounces@lists.gforge.inria.fr">pharo-project-bounces@lists.gforge.inria.fr</a> [mailto:<a href="mailto:pharo-project-bounces@lists.gforge.inria.fr">pharo-project-bounces@lists.gforge.inria.fr</a>] On Behalf Of Igor Stasenko<br>
Sent: Monday, July 19, 2010 3:02 PM<br>
To: Pharo Development<br>
Subject: [Pharo-project] Fwd: [squeak-dev] Re: Debug-it and external calls<br>
<br>
---------- Forwarded message ----------<br>
From: Igor Stasenko <<a href="mailto:siguctua@gmail.com">siguctua@gmail.com</a>><br>
Date: 19 July 2010 23:00<br>
Subject: Re: [squeak-dev] Re: [Pharo-project] Debug-it and external calls<br>
To: The general-purpose Squeak developers list <<a href="mailto:squeak-dev@lists.squeakfoundation.org">squeak-dev@lists.squeakfoundation.org</a>><br>
<br>
<br>
On 19 July 2010 21:42, Andreas Raab <<a href="mailto:andreas.raab@gmx.de">andreas.raab@gmx.de</a>> wrote:<br>
> On 7/19/2010 9:29 AM, Igor Stasenko wrote:<br>
>><br>
>> No, debugger, actually doing a primitive call, but in order to<br>
>> intercept a primitive failure and 'step into' method, it does the<br>
>> trick with replacing the subject method with temporary method (which<br>
>> will call the same primitive with same arguments& recevier).<br>
>> Then, if primitive runs ok, it simply behaves as a 'step over', and<br>
>> if primitive fails, then debugger itercepts it and creates a context<br>
>> for subject method, which should handle primitive failure.<br>
><br>
> Correct. Stepping over FFI calls will only fail if the FFI call<br>
> actually fails. So whatever the problem it's somewhere in your FFI call.<br>
><br>
But! The problem which i discovered not long ago, that if FFI method contains many arguments (close to max 15), then debugger is unable to invoke it, because #perform:... method works in a way, that its using temps of context, where it were invoked. I fixed it by setting a #perform method's frame to be a large one.<br>
<br>
<br>
See <a href="http://bugs.squeak.org/view.php?id=7534" target="_blank">http://bugs.squeak.org/view.php?id=7534</a><br>
<br>
> Cheers,<br>
> - Andreas<br>
><br>
>> On 19 July 2010 19:22, Schwab,Wilhelm K<<a href="mailto:bschwab@anest.ufl.edu">bschwab@anest.ufl.edu</a>> wrote:<br>
>>><br>
>>> So the external call does not happen and the fall-back code executes.<br>
>>> That explains it. It might be nice if FFI calls could be made to<br>
>>> happen, or at least to have a more informative error message, such<br>
>>> as "Primitive not executed by the debugger" rather than "Unable to find function address."<br>
>>><br>
>>> Bill<br>
>>><br>
>>><br>
>>><br>
>>><br>
>>> ________________________________________<br>
>>> From: <a href="mailto:pharo-project-bounces@lists.gforge.inria.fr">pharo-project-bounces@lists.gforge.inria.fr</a><br>
>>> [<a href="mailto:pharo-project-bounces@lists.gforge.inria.fr">pharo-project-bounces@lists.gforge.inria.fr</a>] On Behalf Of Levente<br>
>>> Uzonyi [<a href="mailto:leves@elte.hu">leves@elte.hu</a>]<br>
>>> Sent: Monday, July 19, 2010 12:20 PM<br>
>>> To: <a href="mailto:Pharo-project@lists.gforge.inria.fr">Pharo-project@lists.gforge.inria.fr</a><br>
>>> Subject: Re: [Pharo-project] Debug-it and external calls<br>
>>><br>
>>> On Mon, 19 Jul 2010, Schwab,Wilhelm K wrote:<br>
>>><br>
>>>> I often (not always??) find that using the Debug-it command and<br>
>>>> debugging into an external call (FFI) leads to a false claim of<br>
>>>> "Unable to find function address." Stepping over the call works;<br>
>>>> stepping into it make it look like it fails.<br>
>>>><br>
>>>> I am curently using a 1.1 RC2 image and the 4.0.3 vm on Ubuntu. I<br>
>>>> cannot easily move the offending code to Windows, nor do I<br>
>>>> particularly care to do so (feels good<g>), and FFI is fairly uncommon in Squeak/Pharo.<br>
>>>><br>
>>>> Can anyone else tinker with this to see if it is real?<br>
>>><br>
>>> IIRC when you're using the debugger, primitives are not evaluated.<br>
>>><br>
>>><br>
>>> Levente<br>
>>><br>
>>>><br>
>>>> Bill<br>
>>>><br>
>>>><br>
>>>> _______________________________________________<br>
>>>> Pharo-project mailing list<br>
>>>> <a href="mailto:Pharo-project@lists.gforge.inria.fr">Pharo-project@lists.gforge.inria.fr</a><br>
>>>> <a href="http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project" target="_blank">http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project</a><br>
>>>><br>
>>><br>
>>> _______________________________________________<br>
>>> Pharo-project mailing list<br>
>>> <a href="mailto:Pharo-project@lists.gforge.inria.fr">Pharo-project@lists.gforge.inria.fr</a><br>
>>> <a href="http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project" target="_blank">http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project</a><br>
>>><br>
>>><br>
>><br>
>><br>
>><br>
><br>
><br>
><br>
<br>
<br>
<br>
--<br>
Best regards,<br>
Igor Stasenko AKA sig.<br>
<br>
<br>
<br>
--<br>
Best regards,<br>
Igor Stasenko AKA sig.<br>
<br>
_______________________________________________<br>
Pharo-project mailing list<br>
<a href="mailto:Pharo-project@lists.gforge.inria.fr">Pharo-project@lists.gforge.inria.fr</a><br>
<a href="http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project" target="_blank">http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project</a></div></div></blockquote></div><br>