[Vm-dev] [Pharo-project] is there a way to avoid all the #tryNamedPrimitive:with: * in ProtoObject

Eliot Miranda eliot.miranda at gmail.com
Mon Jan 30 23:45:01 UTC 2012


On Sat, Jan 28, 2012 at 12:52 PM, Igor Stasenko <siguctua at gmail.com> wrote:

>
> On 27 January 2012 19:40, Eliot Miranda <eliot.miranda at gmail.com> wrote:
> >
> >
> >
> > On Fri, Jan 27, 2012 at 9:56 AM, Eliot Miranda <eliot.miranda at gmail.com>
> wrote:
> >>
> >>
> >>
> >> On Fri, Jan 27, 2012 at 5:55 AM, Igor Stasenko <siguctua at gmail.com>
> wrote:
> >>>
> >>>
> >>> On 26 January 2012 19:25, Eliot Miranda <eliot.miranda at gmail.com>
> wrote:
> >>> >
> >>> >
> >>> >
> >>> > On Thu, Jan 26, 2012 at 6:00 AM, Igor Stasenko <siguctua at gmail.com>
> wrote:
> >>> >>
> >>> >>
> >>> >> On 26 January 2012 14:42, Mariano Martinez Peck <
> marianopeck at gmail.com> wrote:
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> > On Thu, Jan 26, 2012 at 11:38 AM, Igor Stasenko <
> siguctua at gmail.com> wrote:
> >>> >> >>
> >>> >> >>
> >>> >> >> On 26 January 2012 11:25, stephane ducasse <
> stephane.ducasse at gmail.com> wrote:
> >>> >> >> >
> >>> >> >> >> phew... done reading through overquoting :)
> >>> >> >> >>
> >>> >> >> >> +1000 to removing tryXYZprimitive:
> >>> >> >> >>
> >>> >> >> >> I was always wondering what those methods for, until i met a
> need to
> >>> >> >> >> support debugging when invoking nativeboost-prim methods,
> >>> >> >> >> because it needs special handing when invoking methods with
> native
> >>> >> >> >> code via debugger.
> >>> >> >> >>
> >>> >> >> >> Then i understood that this mechanism is necessary.. yet a
> bit awkward..
> >>> >> >> >>
> >>> >> >> >> Funny. Even after implementing the fix, I still do not
> understand why all these is needed. Can someone explain to a newbie why
> invoking primitives (whether they are normal primitives, named primitives
> or NB primitives) from the debugger is different than invoking them
> normally (as when they are invoked by normal code)
> >>> >> >> >
> >>> >> >> > Yes I want to understand too.
> >>> >> >> >
> >>> >> >>
> >>> >> >> Here the hint for you:
> >>> >> >>  - what should happen when you doing 'step in' on method, which
> has a primitive?
> >>> >> >>
> >>> >> >> Apparently, it should invoke that primitive , otherwise you will
> have
> >>> >> >> difference between running and debugging modes,
> >>> >> >> and will have different results, which makes debugger useless.
> >>> >> >> So debugger should detect "somehow" if primitive was failed, and
> then
> >>> >> >> step in into given method,
> >>> >> >> or if its not, then step in = step over.
> >>> >> >>
> >>> >> >> And these 'tryXYZ ... ' is exactly for solving this dilemma.
> >>> >> >
> >>> >> >
> >>> >> > Thanks Igor. So I wrote what I understood. Problem is that I
> always write for newbies (like me) so if it is too obvios or too long to
> put it as comment, let me know.
> >>> >> > I would appreaciate if someone can validate what I wrote.
> >>> >> > Today I will create a slice with mentioned solution + comments +
> removal of all those tryNamedPrimitive*.
> >>> >> >
> >>> >> > ----
> >>> >> >
> >>> >> > "When using the debugger we want to run a method step by step.
> But what happens when we do a step into a CompiledMethod which has a
> primitive? If such a method is executed form outside the Debugger (normal
> scenario) the VM knows that such CompiledMethod has a primitive declaration
> and hence executes it. If it fails, then it continues executing all the
> bytecodes of the method. Otherwise, it just returns.
> >>> >> >
> >>> >> > Now, what is the problem with the Debugger? The problem is that
> if the primitive fail, we don't want that the VM directly executes all the
> remaining bytecodes of the method. Instead, we would like to go step by
> step with he Debugger, just as happens with normal methods.
> >>> >> >
> >>> >> > To solve the mentioned problem, we use the following trick: We
> have the original compiled method (the one that has a primitive
> invocation), the receiver and the arguments. So the idea is to use a
> template compiled method that ONLY contains the primitive declaration (it
> doesn't include all the original smalltalk code after the primitive).
> #tryNamedPrimitiveTemplateMethod answers such a template method which looks
> like:
> >>> >> >
> >>> >> > tryNamedPrimitive
> >>> >> >     <primitive:'to be set later' module:'to be ser later'>
> >>> >> >     ^ ContextPart primitiveFailToken'
> >>> >> >
> >>> >> >  Since this method does not change its bytecodes for every
> invocation, we can reuse it for all methods with primitives. There are only
> 2 things we have to change in the template: the number of arguments and the
> primitive declaration (to use the correct primitive name and module name).
> >>> >> >
> >>> >> > Then what we do is to run that compiled method with the receiver
> and arguments we have. The result is that we will be invoking almost the
> same original method but a slightly different version that does not have
> the smalltalk part after the primitive and that in contrast is sends
> #primitiveFailToken (which tells the Debugger what to do after). If this
> method invocation does not fail, then the Debugger continues debugging the
> sender of the primitive method. In this case, the step in is the same as
> step over. If the primitive fails, then the debugger continues executing
> the smalltalk part after the primitive method. In this case, step in is a
> real step in.  "
> >>> >> >
> >>> >> >
> >>> >> There's another problem with using "tryXYZ" when primitive doing
> >>> >> something weird with contexts (like block's #value)
> >>> >> because then debugger cannot intercept switching contexts so easily,
> >>> >> when primitive doing manipulation with contexts,
> >>> >> because debugger have no idea where to put his next "break point" to
> >>> >> do step-by-step evaluation.
> >>> >>
> >>> >> I thinking that there should be special primitive which:
> >>> >>  - takes a context, a receiver, a method and arguments (or just a
> >>> >> context, if debugger ensures to pass it in prepared state i.e.
> >>> >> receiver and args already on stack)
> >>> >>  - invokes method's primitive
> >>> >>  - answers a nil if primitive failed or a new context object, which
> >>> >> holds an updated context state after primitive possibly manipulated
> >>> >> with context(s)
> >>> >
> >>> >
> >>> > No, not needed.  Manipulation of contexts does not require
> primitives since contexts are first-class, and the effect of these
> primitives can be simulated in Smalltalk (just as the effect of executing
> bytecodes can be simulated).  Look at the caller of tryPrimitive:withArgs:
> and tryNamedPrimitiveIn:for:withArgs:, namely
> ContextPart>doPrimitive:method:receiver:args:.  It handles all these
> primitives such as perform:[*], value[:value:*], withArgs:executeMethod:.
> >>>
> >>> Looks like you don't read carefully. Exactly because of looking to
> >>> ContextPart>doPrimitive:method:receiver:args: i proposed another way.
> >>> And besides it is much simpler.
> >>> If tomorrow we introduce new primitive which manipulates context, you
> >>> will need to fix this method,
> >>> because it duplicating knowledge of what VM does.
> >>
> >>
> >> I disagree.  ContextPart already duplicates the VM.  The direction you
> propose ends up putting all context functionality in the VM, which makes
> the system harder to understand (the SMalltalk code is more readable),
> harder to experiment with (one can no longer implement experiments in the
> image, for example as I did in implementing the closure compiler), and
> harder to implement (implementing and testing tricky do-lots-of-things
> primitives in the VM is harder than implementing and testing their
> equivalents in the image).  So I think this primitive is a really bad idea.
>  Its going to be much more complex than doPrimitive:method:receiver:args:.
>
> Hmm.. are you sure about much complex?
> As to me it is fairly simple:
>
> 1. remember current context (debugger)
> 2. activate a debuggable context (passed as argument)
> 3. invoke a primitive
> 4. activate a debugger context back and return a modified debuggable
> context as result.
>

How can you just invoke perform: without having the perform primitive
execute Smalltalk?  What one wants is the primitive to get one to the next
context, /not/ to do all that the primitive does.  Primitive perform: does
a full send and evaluation, not just the lookup and initial dispatch.
 Block value, evaluates the block, not just activates it.  The code in
doPrimitive:... just does the "get me to the next context" bit.


> > And of course experience shows it is much quicker and easier to
> distribute Smalltalk improvements than new virtual machines, so pushing
> important functionality down into the VM is a retrograde step.  It seems
> out of keeping with NativeBoost and ideas such as moving the linking
> machinery for named primitives and the FFI up into the image, decomposing
> them into a more general set of smaller simpler primitives.
> >
>
> This is true.  But exactly because of it, we need such prim!
> Think, how you going to debug a dynamically generated code, which
> manipulates a contexts,
> if by invoking it, your debugger will immediately lose a control?
>
> A simulation is not an answer here, because you actually want to run
> your your dynamically generated code and check if it behaves well,
> while if you just simulate it you will never know if its ok, because
> then you will be testing whether your simulation behaves correctly,
> instead of real thing.
>

I don;t think this is right.  One wants to run only the initial part of the
primitive, not the subsequent execution it spawns.


> And of course, then every time you changing the native implementation,
> you will also need to update a simulation code as well -> a best way
> to kill a project, when two independent parts must behave identically,
> but since a lazy developers forgetting keep them in sync, it falling
> apart.
>

Biut that didn't happen when I introduced BlockClosure did it?


>
> This is actually a situation which we having now, because implementation of
> #doPrimitive: primitiveIndex method: meth receiver: receiver args:
> arguments
> is based on assumption that VM are in sync with image, and every time
> you change something, you have to go there and sync stuff.
> Why we cannot follow DRY principle?
>

Because one isn't repeating oneself, given that the actual execution
primitives differ from the versions that do just the first bit.


>
> So, without having it, i had to introduce an awful hack in
> NativeBoost, an override in ContextPart package,
> because i have no choice:
>

The test has to go somewhere...  You could perhaps make it neater if you
put it in doPrimitive:.


> ContextPart>>tryNamedPrimitiveIn: aCompiledMethod for: aReceiver
> withArgs: arguments
>         "Hack. Attempt to execute the named primitive from the given
> compiled method"
>         | selector theMethod spec |
>
>        (NativeBoost isNativeMethod: aCompiledMethod) ifTrue: [
>                ^ NativeBoost tryRunNativeCode: aCompiledMethod for:
> aReceiver
>                        withArgs: arguments onFail: [ ^ PrimitiveFailToken
> ].
>                ].
>        arguments size > 8 ifTrue:[^PrimitiveFailToken].
>         selector := #(
>                tryNamedPrimitive
>                tryNamedPrimitive:
>                tryNamedPrimitive:with:
>                tryNamedPrimitive:with:with:
>                tryNamedPrimitive:with:with:with:
>                tryNamedPrimitive:with:with:with:with:
>                tryNamedPrimitive:with:with:with:with:with:
>                tryNamedPrimitive:with:with:with:with:with:with:
>                tryNamedPrimitive:with:with:with:with:with:with:with:) at:
> arguments size+1.
>         theMethod := aReceiver class lookupSelector: selector.
>        theMethod == nil ifTrue:[^PrimitiveFailToken].
>         spec := theMethod literalAt: 1.
>        spec replaceFrom: 1 to: spec size with: (aCompiledMethod literalAt:
> 1) startingAt: 1.
>         theMethod flushCache.
>        selector flushCache.
>        ^aReceiver perform: selector withArguments: arguments
>
> ---------------
> NativeBoost class>>tryRunNativeCode: aCompiledMethod for: aReceiver
> withArgs: arguments onFail: aFailBlock
>        "run only a native code of given method,
>        and if it fails evaluate aFailBlock instead"
>        | src node m failObj result |
>        self assert: (self isNativeMethod: aCompiledMethod).
>
>        src := String streamContents: [:str |
>                str nextPutAll: 'NBDebug_stub'.
>                1 to: aCompiledMethod numArgs do: [:i  |
>                        i = 1
>                                ifTrue: [ str nextPutAll: ': ' ]
>                                ifFalse: [ str nextPutAll: ' with: ' ].
>                        str nextPutAll: 'arg'.
>                        i printOn: str.
>                ].
>                str cr;
>                nextPutAll: ' <primitive: #primitiveNativeCall module:
> #NativeBoostPlugin >'; cr;
>                nextPutAll: ' ^ ''theFailLiteral'' '.
>        ].
>
>        node := Compiler new
>                compile: src
>                in: aCompiledMethod methodClass
>                classified: nil
>                notifying: nil
>                ifFail: [ ^ aFailBlock value ].
>
>        m := node generate: aCompiledMethod trailer.
>        self assert: (m literalAt: 2) = 'theFailLiteral'.
>        failObj := Object new.
>        m literalAt: 2 put: failObj.
>        m flushCache.
>        m selector flushCache.
>        result := m valueWithReceiver: aReceiver arguments: arguments.
>
>        ^ result == failObj ifTrue: [ aFailBlock value] ifFalse: [ result ]
>
> and as you can see, an implementation of "please simply run just the
> primitive of method"
> at language side is not looking nice and easy.
> And of course, if by chance , a native code manipulate context, then
> the above hack not going to work anymore. This code actually assumes
> that primitive plays nicely and not doing such nasty things :)
>

Right, and if NB starts replacing significant parts of execution then
you'll want to simulate NB code, not simply execute it when in the
debugger.  Primitives are just that, primitives.  They can't be invoked
willy-nilly.  They don't obey the same constraints as Smalltalk code and
they'll run away if you're not careful.  doPrimitive: is careful, except
for FFI calls that do callbacks (one can't step into a callback).  Likewise
one can't step into a NB primitive that calls-back.  Tricky.



> --
> Best regards,
> Igor Stasenko.
>



-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20120130/e84c2616/attachment-0001.htm


More information about the Vm-dev mailing list