[Vm-dev] RE: [Pharo-project] Fwd: [squeak-dev] Re: Debug-it and external calls

Eliot Miranda eliot.miranda at gmail.com
Mon Jul 19 21:39:59 UTC 2010


On Mon, Jul 19, 2010 at 1:10 PM, Schwab,Wilhelm K <bschwab at anest.ufl.edu>wrote:

>
> If the FFI call is flawed, why does it work when not stepping in the
> debugger?  I'm not buying it :)
>

A few people have seen problems with the part of the debugger in question.
 Here's the canonical version of the method by Andreas 'ar 5/25/2000 20:41'.
 It is invoked from ContextPart>>doPrimitive:method:receiver:args: and its
job is to invoke just the primitive and either answer the result or a
special PrimitiveFailed token.  The debugger then checks for the result
being the PrimitiveFailed token and if so steps into the method.

ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs:
arguments
"Hack. Attempt to execute the named primitive from the given compiled
method"
| selector theMethod spec |
arguments size > 8 ifTrue:[^PrimitiveFailToken].
selector _ #(
tryNamedPrimitive
tryNamedPrimitive:
tryNamedPrimitive:with:
tryNamedPrimitive:with:with:
tryNamedPrimitive:with:with:with:
tryNamedPrimitive:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:with:) at: arguments size+1.
theMethod _ aReceiver class lookupSelector: selector.
theMethod == nil ifTrue:[^PrimitiveFailToken].
spec _ theMethod literalAt: 1.
spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt: 1)
startingAt: 1.
^aReceiver perform: selector withArguments: arguments

What this does is slam the named primitive to be evaluated into the relevant
implementation
of tryNamedPrimitive[:with:with:with:with:with:with:with:] and invoke it.
 One minor problem is that there are only 9 methods so there's only support
for 0 to 8 arguments.  The real problem with this is that it doesn't flush
the VM's method lookup cache and so one can end up invoking a different
named primitive, one that was installed into the same method by a previous
invocation of tryNamedPrimitiveIn:for:withArgs:.

Michael Haupt saw the effect of this doing bytecode simulation on Cog.  For
some reason it's much more likely to hit the cache flush issue in Cog tan in
the standard interpreter (probably because Cog makes less use of the method
cache because machine code sends have their send caches inline in the
machine code).

I fixed this for Michael by using the following version 'eem 5/23/2010
14:13'
ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs:
arguments
"Hack. Attempt to execute the named primitive from the given compiled
method"
| selector theMethod spec |
arguments size > 8 ifTrue:[^PrimitiveFailToken].
selector := #(
tryNamedPrimitive
tryNamedPrimitive:
tryNamedPrimitive:with:
tryNamedPrimitive:with:with:
tryNamedPrimitive:with:with:with:
tryNamedPrimitive:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:with:) at: arguments size+1.
theMethod := aReceiver class lookupSelector: selector.
theMethod == nil ifTrue:[^PrimitiveFailToken].
spec := theMethod literalAt: 1.
spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt: 1)
startingAt: 1.
theMethod flushCache.
selector flushCache.
^aReceiver perform: selector withArguments: arguments

It adds "theMethod flushCache. selector flushCache" to flush the cache and
at least for a couple of months things seemed to be OK.  But Michael sent me
a different case a week ago that turns out to be due to exactly the same
cause.  When simulated his code was trying to invoke
the primDigitDivNegative primitive via digitDiv:neg: (which doesn't exist in
Cog and so should simply fail) but it was succeeding, running some bogus
primitive.

There is a more fundamental issue with the above.  try debugging it.  If you
do you stand a good chance of the debugger overwriting what the debugger
version of the method is trying to effect.  That's why if you look at the
analogous machinery for num bered primitives you'll see the above approach
(which was used in Smalltalk-80 V2) is discarded and a much more reliable
approach, a primitive-executing primitive, is used, namely
ProtoObject>>#tryPrimitive:withArgs: 'ajh 1/31/2003 22:21':
ProtoObject>>tryPrimitive: primIndex withArgs: argumentArray
"This method is a template that the Smalltalk simulator uses to
execute primitives. See Object documentation whatIsAPrimitive."

<primitive: 118>
^ ContextPart primitiveFailToken

In the Teleplace images I've taken a similar approach, providing a primitive
in Cog.  The below is slightly different because the Teleplace images
debugger supports primitive error codes (and I need to fold this back into
Squeak asap). ContextPart>>tryNamedPrimitiveIn:for:withArgs: eem 5/12/2009

ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver withArgs:
arguments
| selector theMethod spec receiverClass |
<primitive: 218 error: ec>
ec ifNotNil:
["If ec is an integer other than -1 there was a problem with primitive 218,
  not with the external primitive itself.  -1 indicates a generic failure
(where
  ec should be nil) but ec = nil means primitive 218 is not implemented.  So
  interpret -1 to mean the external primitive failed with a nil error code."
 ec isInteger ifTrue:
[ec = -1
ifTrue: [ec := nil]
ifFalse: [self primitiveFailed]].
^{PrimitiveFailToken. ec}].
"Assume a nil error code implies the primitive is not implemented and fall
back on the old code."
"Hack. Attempt to execute the named primitive from the given compiled
method"
arguments size > 8 ifTrue:
[^{PrimitiveFailToken. nil}].
selector := #(
tryNamedPrimitive
tryNamedPrimitive:
tryNamedPrimitive:with:
tryNamedPrimitive:with:with:
tryNamedPrimitive:with:with:with:
tryNamedPrimitive:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:
tryNamedPrimitive:with:with:with:with:with:with:with:) at: arguments size+1.
receiverClass := self objectClass: aReceiver.
theMethod := receiverClass lookupSelector: selector.
theMethod == nil ifTrue:
[^{PrimitiveFailToken. nil}].
spec := theMethod literalAt: 1.
spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt: 1)
startingAt: 1.
Smalltalk unbindExternalPrimitives.
^self object: aReceiver perform: selector withArguments: arguments inClass:
receiverClass

With this approach there's no argument count limit and there's no chance of
executing the wrong primitive.  But it requires a primitive in the VM.
 Perhaps the VM folks can take a look at porting the Cog
primitive primitiveDoNamedPrimitiveWithArgs back to the base VM and then we
can use something like the above with the fallback code, liberally sprinkled
with cache flushes, only as a fallback.

HTH
Eliot


>
>
>
> -----Original Message-----
> From: pharo-project-bounces at lists.gforge.inria.fr [mailto:
> pharo-project-bounces at lists.gforge.inria.fr] On Behalf Of Igor Stasenko
> Sent: Monday, July 19, 2010 3:02 PM
> To: Pharo Development
> Subject: [Pharo-project] Fwd: [squeak-dev] Re: Debug-it and external calls
>
> ---------- Forwarded message ----------
> From: Igor Stasenko <siguctua at gmail.com>
> Date: 19 July 2010 23:00
> Subject: Re: [squeak-dev] Re: [Pharo-project] Debug-it and external calls
> To: The general-purpose Squeak developers list <
> squeak-dev at lists.squeakfoundation.org>
>
>
> On 19 July 2010 21:42, Andreas Raab <andreas.raab at gmx.de> wrote:
> > On 7/19/2010 9:29 AM, Igor Stasenko wrote:
> >>
> >> No, debugger, actually doing a primitive call, but in order to
> >> intercept a primitive failure and 'step into' method, it does the
> >> trick with replacing the subject method with temporary method (which
> >> will call the same primitive with same arguments&  recevier).
> >> Then, if primitive runs ok, it simply behaves as a 'step over', and
> >> if primitive fails, then debugger itercepts it and creates a context
> >> for subject method, which should handle primitive failure.
> >
> > Correct. Stepping over FFI calls will only fail if the FFI call
> > actually fails. So whatever the problem it's somewhere in your FFI call.
> >
> But! The problem which i discovered not long ago, that if FFI method
> contains many arguments (close to max 15), then debugger is unable to invoke
> it, because #perform:... method works in a way, that its using temps of
> context, where it were invoked. I fixed it by setting a #perform method's
> frame to be a large one.
>
>
> See http://bugs.squeak.org/view.php?id=7534
>
> > Cheers,
> >  - Andreas
> >
> >> On 19 July 2010 19:22, Schwab,Wilhelm K<bschwab at anest.ufl.edu>  wrote:
> >>>
> >>> So the external call does not happen and the fall-back code executes.
> >>>  That explains it.  It might be nice if FFI calls could be made to
> >>> happen, or at least to have a more informative error message, such
> >>> as "Primitive not executed by the debugger" rather than "Unable to find
> function address."
> >>>
> >>> Bill
> >>>
> >>>
> >>>
> >>>
> >>> ________________________________________
> >>> From: pharo-project-bounces at lists.gforge.inria.fr
> >>> [pharo-project-bounces at lists.gforge.inria.fr] On Behalf Of Levente
> >>> Uzonyi [leves at elte.hu]
> >>> Sent: Monday, July 19, 2010 12:20 PM
> >>> To: Pharo-project at lists.gforge.inria.fr
> >>> Subject: Re: [Pharo-project] Debug-it and external calls
> >>>
> >>> On Mon, 19 Jul 2010, Schwab,Wilhelm K wrote:
> >>>
> >>>> I often (not always??) find that using the Debug-it command and
> >>>> debugging into an external call (FFI) leads to a false claim of
> >>>> "Unable to find function address."  Stepping over the call works;
> >>>> stepping into it make it look like it fails.
> >>>>
> >>>> I am curently using a 1.1 RC2 image and the 4.0.3 vm on Ubuntu.  I
> >>>> cannot easily move the offending code to Windows, nor do I
> >>>> particularly care to do so (feels good<g>), and FFI is fairly uncommon
> in Squeak/Pharo.
> >>>>
> >>>> Can anyone else tinker with this to see if it is real?
> >>>
> >>> IIRC when you're using the debugger, primitives are not evaluated.
> >>>
> >>>
> >>> Levente
> >>>
> >>>>
> >>>> Bill
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Pharo-project mailing list
> >>>> Pharo-project at lists.gforge.inria.fr
> >>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >>>>
> >>>
> >>> _______________________________________________
> >>> Pharo-project mailing list
> >>> Pharo-project at lists.gforge.inria.fr
> >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >>>
> >>>
> >>
> >>
> >>
> >
> >
> >
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> Pharo-project at lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20100719/c3b20cdd/attachment.htm


More information about the Squeak-dev mailing list