[squeak-dev] Re: [Vm-dev] Better VM <-> plugin API

Igor Stasenko siguctua at gmail.com
Sat Nov 22 19:06:45 UTC 2008


2008/11/22 Eliot Miranda <eliot.miranda at gmail.com>:
>
>
> On Sat, Nov 22, 2008 at 9:34 AM, Igor Stasenko <siguctua at gmail.com> wrote:
>>
>> 2008/11/22 Andreas Raab <andreas.raab at gmx.de>:
>> > Igor Stasenko wrote:
>> >>
>> >> By more security, i meant a little feature, that since getting atom
>> >> value is inexpensive operation, you can
>> >> load value identified by atom and check for non-null value each time
>> >> before calling function.
>> >
>> > Oh, I think you mean it can be safer to use (with which I would agree)
>> > not
>> > necessarily more secure.
>> >
>> >> Old scheme, needs to check for non-null as well, but in addition you
>> >> need to be careful to clean out this pointer when you get notification
>> >> that module, from where you taken this pointer is unloaded.
>> >
>> > Yes. Although, you won't get around doing something about unloading
>> > either -
>> > we spent today learning about the intricacies of unloading OpenAL32.dll
>> > and
>> > it turned out to be absolutely crucial to be able to unload our own
>> > plugins.
>> > A flat shared namespace might have caused some serious issues here.
>> >
>> Almost anything "might have cause some serious issues", if you don't
>> use it wisely (Class become: nil).
>> Its not an argument not to use it.
>>
>> >> I'm not sure what you mean by per-plugin namespace.
>> >> And how much difference in namespaces between this:
>> >>  bitblt = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin")
>> >> and this:
>> >>   atom = makeAtom("BitBltPlugin.ioBitBlt");
>> >>   bitBlt = getAtomValue(atom);
>> >>
>> >> The difference that first binds symbol at compile/link time, while
>> >> second - at run time.
>> >
>> > It would be quite possible to bind this at runtime, too. But what I mean
>> > by
>> > per-plugin namespace is that the *export* isn't done into a flat
>> > namespace
>> > but rather into a structured one.
>> >
>> > And yes, one could conceivably use a naming scheme that is equivalent in
>> > practice but unless that's automated (the current scheme is fully
>> > automated)
>> > it seems error-prone and one of these things were people are simply too
>> > lazy
>> > in practice to utilize it correctly (if that were different I would
>> > expect
>> > people to use class and selector prefixes consistently which they
>> > don't).
>> >
>> It could be automated or used manually. Its open for any intrusion :)
>>
>> >> As for modularity, let me illustrate what i meant.
>> >> There are many primitives in VM, which look like:
>> >>
>> >> primitiveFoo: x with: y
>> >>
>> >> ^ self cCode: 'ioFoo(x,y)'.
>> >>
>> >> Obviously, when you generate this code using VMMaker, it wont compile
>> >> unless you having ioFoo() function implemented somewhere.
>> >> And you have a little choice where to put this implementation - in one
>> >> of internal plugins or in platform-specific part of VM.
>> >>
>> >> Now, lets consider, if i refactor this code to use atoms (i skipping
>> >> details like declarations etc) :
>> >>
>> >> primitiveFoo: x with: y
>> >>
>> >> ioFoo := getAtomValue: AtomIoFoo.
>> >> ioFoo notNil ifTrue: [ ^ self cCode: 'ioFoo(x,y)' ].
>> >> ^ self primitiveFail.
>> >>
>> >> 1. the code will compile regardless i declared a function or not.
>> >> 2. i'm free in choice where to put the implementation of this
>> >> function, or even build VM initially w/o this function.
>> >> 3. across different platforms, one can use same VMMaker to generate
>> >> sources , and it will always compile & link.
>> >>
>> >> doesn't it look like more modular?
>> >
>> > No. It looks utterly pointless to me. You introduce a plugin that does
>> > nothing but looking up and call an atom; what good is that plugin? If
>> > you
>> > generalize that just a little you have the FFI where you might declare
>> > ioFoo() directly and call it. Which of course could be done via atom
>> > table
>> > too, but I still fail to see how that would be more modular.
>> >
>>
>> Not a plugin. I mentioned VM code in interp.c. I see little need in
>> having such constructs in plugin.
>> In VM core, however, there is always a bit of uncertainty - VM forced
>> to provide primitive by default (because its a standart one), but on
>> different platforms, depending on their capabilities or your intent,
>> you may omit putting functionality of some primitives into VM.
>> This is where such scheme is can be quite useful - instead of stubbing
>> function in sources, or exploring what function has to return to make
>> primitive fail , just remove it from build.
>>
>> >> Now, take a look at a sqWin32Stubs.c - how it would look if we would
>> >> use shared namespace? It would look as zero length file.
>> >> Because all we have to do in platform code is just initialize pointers:
>> >>
>> >> pointers = {
>> >>  "ioFoo" , ioFoo
>> >> #ifdef NO_SOME_STUFF
>> >>  "ioBar" , ioBar
>> >> #endif
>> >>  ...
>> >> };
>> >
>> > And if you stick this in sqWin32Stubs.c (or its equivalent) you end up
>> > with
>> > a non-empty stubs file. In other words, you are replacing one set of
>> > stubs
>> > with another one. Not much of an improvement.
>> >
>>
>> Its not the same thing. Currently you have to define a function - with
>> implementation or with empty body to make compiler content.
>> In example i showed - you can do simpler - just omit binding a
>> function to a symbol.
>> Suppose we having a well structured organization of VM platform
>> sources,  then it might look like:
>>
>> #ifdef SUPPORT_FILES
>> #include "file_functions.inc"
>> /// or put all functions here w/o using #include
>> #endif
>>
>> instead of something like:
>>
>> #ifdef SUPPORT_FILES
>> #include "file_functions.inc"
>> /// or put all functions here w/o using #include
>> #else
>> #include "file_functions_stubs.inc"
>> #endif
>>
>> >> I'm currently trying to make a HostWindowsPlugin and want to put all
>> >> windowing and i/o stuff in it from VM platform sources.
>> >
>> > Why do another one? Is there something that the current host window
>> > plugin
>> > doesn't address?
>>
>> Well, there's many things.
>> Don't want to OT here, just simple example: when image fires up, i
>> want to show a splash screen, and then, when user clicks on it, or
>> some time passed , hide it and show the "main" window. ALL is
>> controlled from image, of course , e.g. one might want to show splash
>> screen and when user clicks on it - just quit the squeak :)
>>
>> >
>> >> It would be much a cut'n'paste experience to me, if all callouts in VM
>> >> would use atoms - because then i don't need to touch headers & sources
>> >> and many different places to make compiler content. Because there
>> >> would be no need in exporting function in C-like manner.
>> >
>> > Ah. But that's a fallacy. That you don't need to "touch" these places
>> > doesn't mean you don't need to know about them. In fact, having to touch
>> > them, having the compiler complain about them is a great way to learn
>> > and
>> > know and understand all the areas you need to touch - if the VM would
>> > randomly crash on you because you have replaced one but not another
>> > function
>> > you'd be in for a hellish experience. Yes, the C compiler can be
>> > notorious
>> > but it can also be a pretty good teacher.
>> >
>>
>> The same self-fallacy is to be confident, that if your code compiles
>> ok it doesn't crash in some random place :)
>>
>> >> And lastly, remember InterpreterProxy thing? Each time you want to
>> >> introduce new functionality , you have to extend the structure and
>> >> increase version number. But what is it? Its just a list of pointers!
>> >
>> > No, it's not. It's an interface (a contract) between the VM and the
>> > plugin.
>> >
>> >> Isn't it would be simpler for plugin to just lookup a function by its
>> >> name - and use it if such function exists. Its pretty much same as
>> >> dynamic linking, just s/dlsym/getAtomValue/ and don't let your code be
>> >> constrained by compiler/linker anymore :)
>> >
>> > I *very much* doubt that it would be simpler for each plugin to look up
>> > the
>> > function every time they use it and test for its existence every time it
>> > is
>> > used. Consider this primitive:
>> >
>> > primitiveStringClone
>> >  interpreterProxy methodArgument = 1
>> >    ifFalse:[^interpreterProxy primitiveFail].
>> >  arg := interpreterProxy stackValue: 0.
>> >  (interpreterProxy isBytes: arg)
>> >    ifFalse:[^interpreterProxy primitiveFail].
>> >  clone := interpreterProxy clone: arg.
>> >  interpreterProxy pop: 2.
>> >  interpreterProxy push: clone.
>> >
>> > Now let's rewrite this in pseudo-code to see what it would look like
>> > without
>> > an interface:
>> >
>> > primitiveStringClone
>> >  ((interpreterProxy has: #methodArgumentCount)
>> >    and:[interpreterProxy methodArgument = 1])
>> >      ifFalse:[^interpreterProxy primitiveFail].
>> >  (interpreterProxy has: #stackValue)
>> >      ifFalse:[^interpreterProxy primitiveFail].
>> >  arg := interpreterProxy stackValue: 0.
>> >  (interpreterProxy has: #isBytes)
>> >      ifFalse:[^interpreterProxy primitiveFail].
>> >  (interpreterProxy isBytes: arg)
>> >      ifFalse:[^interpreterProxy primitiveFail].
>> >  (interpreterProxy has: #clone)
>> >      ifFalse:[^interpreterProxy primitiveFail].
>> >  clone := interpreterProxy clone: arg.
>> >  (interpreterProxy has: #pop)
>> >      ifFalse:[^interpreterProxy primitiveFail].
>> >  interpreterProxy pop: 2.
>> >  (interpreterProxy has: #push)
>> >      ifFalse:[^interpreterProxy primitiveFail].
>> >  interpreterProxy push: clone.
>> >
>> >
>> > Simpler? You've got to be kidding me ;-)
>> >
>>
>> This counter-example missing a point :)
>> Take a look at Hydra code, where InterpreterProxy has left with only 3
>> functions, and will stay to have 3 functions forever without need in
>> extending protocol , because rest functions is obtained dynamicaly
>> using name lookup.
>> A plugin simply refuse to start without having all requested VM
>> functions be non-null. But the trick is, that VM not forced anymore to
>> support obsolete stuff of ever-growing InterpreterProxy protocol.
>
> I like this very much.  Not totally, just very much :)
> What's good here is that the plugin can access functions directly in the VM
> and not go through the slow interpreterProxy.  Because C supports syntax for
> calls through function pointers that looks just like normal functions calls
> wouldn't look very different to normal calls.  The function pointers simply
> need to be decared.

right!
> But what I don't like is having interpreterProxy persist at all. Why not get
> rid of it except for initialization and pass it in through setInterpreter?
>  What do you need interpreterProxy to persist for?
right!
> So Slang generates a standard prolog for all the functions used in a plugin
> at the beginning of the file (nice documentation; lists exactly the
> functions the plugin uses).  setInterpreter (better called
> initializePlugin?) takes an argument that provides a function to lookup
> function pointers by name.  Slang generates the code to initialize these
> fnction pointers.  Uses of the function pointers look just like normal
> calls.
Right!
> Then internal plugins can be linked directly against the VM.  e.g.
>

Things works exactly in that way in Hydra. :)

> #if EXTERNAL_PLUGIN
> static void (*popthenPush(sqInt,sqInt);
> static void (*primitiveFailed)(void);
> #endif
> ...
> void somePrimitive()
> {
>     ....
>     if (!ok) {
>         primitiveFailed();
>         return;
>     }
>     popthenPush(result);
> }
> ...
> #if EXTERNAL_PLUGIN
> static sqInt
> setInterpreter(InterpreterProxy *interpreterProxy)
> {
>     if (interpreterProxy.oopSize() != 4)
>         return InitErrorWrongOopSize;
>     if (!interpreterProxy.getFunction("popthenPush", &popthenPush)
>      || !interpreterProxy.getFunction("primitiveFailed", & primitiveFailed))
>         return InitErrorMissingFunction;
>     return 0;
> }
> #endif /* EXTERNAL_PLUGIN */
>>
>>
>> > Cheers,
>> >  - Andreas
>> >
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>
>
>
>
>



-- 
Best regards,
Igor Stasenko AKA sig.



More information about the Squeak-dev mailing list