OK, good :) So Igor, what are the three functions interpreterProxy still has?<div><br></div><div>TIA<br><br></div><div>(P.S. this can be backward-compatible, right? We can still provide the old nterface for a while to allow external plugins to still load)</div>
<div><br><div class="gmail_quote">On Sat, Nov 22, 2008 at 11:06 AM, Igor Stasenko <span dir="ltr"><<a href="mailto:siguctua@gmail.com">siguctua@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
2008/11/22 Eliot Miranda <<a href="mailto:eliot.miranda@gmail.com">eliot.miranda@gmail.com</a>>:<br>
<div><div></div><div class="Wj3C7c">><br>
><br>
> On Sat, Nov 22, 2008 at 9:34 AM, Igor Stasenko <<a href="mailto:siguctua@gmail.com">siguctua@gmail.com</a>> wrote:<br>
>><br>
>> 2008/11/22 Andreas Raab <<a href="mailto:andreas.raab@gmx.de">andreas.raab@gmx.de</a>>:<br>
>> > Igor Stasenko wrote:<br>
>> >><br>
>> >> By more security, i meant a little feature, that since getting atom<br>
>> >> value is inexpensive operation, you can<br>
>> >> load value identified by atom and check for non-null value each time<br>
>> >> before calling function.<br>
>> ><br>
>> > Oh, I think you mean it can be safer to use (with which I would agree)<br>
>> > not<br>
>> > necessarily more secure.<br>
>> ><br>
>> >> Old scheme, needs to check for non-null as well, but in addition you<br>
>> >> need to be careful to clean out this pointer when you get notification<br>
>> >> that module, from where you taken this pointer is unloaded.<br>
>> ><br>
>> > Yes. Although, you won't get around doing something about unloading<br>
>> > either -<br>
>> > we spent today learning about the intricacies of unloading OpenAL32.dll<br>
>> > and<br>
>> > it turned out to be absolutely crucial to be able to unload our own<br>
>> > plugins.<br>
>> > A flat shared namespace might have caused some serious issues here.<br>
>> ><br>
>> Almost anything "might have cause some serious issues", if you don't<br>
>> use it wisely (Class become: nil).<br>
>> Its not an argument not to use it.<br>
>><br>
>> >> I'm not sure what you mean by per-plugin namespace.<br>
>> >> And how much difference in namespaces between this:<br>
>> >> bitblt = ioLoadFunctionFrom("ioBitBlt", "BitBltPlugin")<br>
>> >> and this:<br>
>> >> atom = makeAtom("BitBltPlugin.ioBitBlt");<br>
>> >> bitBlt = getAtomValue(atom);<br>
>> >><br>
>> >> The difference that first binds symbol at compile/link time, while<br>
>> >> second - at run time.<br>
>> ><br>
>> > It would be quite possible to bind this at runtime, too. But what I mean<br>
>> > by<br>
>> > per-plugin namespace is that the *export* isn't done into a flat<br>
>> > namespace<br>
>> > but rather into a structured one.<br>
>> ><br>
>> > And yes, one could conceivably use a naming scheme that is equivalent in<br>
>> > practice but unless that's automated (the current scheme is fully<br>
>> > automated)<br>
>> > it seems error-prone and one of these things were people are simply too<br>
>> > lazy<br>
>> > in practice to utilize it correctly (if that were different I would<br>
>> > expect<br>
>> > people to use class and selector prefixes consistently which they<br>
>> > don't).<br>
>> ><br>
>> It could be automated or used manually. Its open for any intrusion :)<br>
>><br>
>> >> As for modularity, let me illustrate what i meant.<br>
>> >> There are many primitives in VM, which look like:<br>
>> >><br>
>> >> primitiveFoo: x with: y<br>
>> >><br>
>> >> ^ self cCode: 'ioFoo(x,y)'.<br>
>> >><br>
>> >> Obviously, when you generate this code using VMMaker, it wont compile<br>
>> >> unless you having ioFoo() function implemented somewhere.<br>
>> >> And you have a little choice where to put this implementation - in one<br>
>> >> of internal plugins or in platform-specific part of VM.<br>
>> >><br>
>> >> Now, lets consider, if i refactor this code to use atoms (i skipping<br>
>> >> details like declarations etc) :<br>
>> >><br>
>> >> primitiveFoo: x with: y<br>
>> >><br>
>> >> ioFoo := getAtomValue: AtomIoFoo.<br>
>> >> ioFoo notNil ifTrue: [ ^ self cCode: 'ioFoo(x,y)' ].<br>
>> >> ^ self primitiveFail.<br>
>> >><br>
>> >> 1. the code will compile regardless i declared a function or not.<br>
>> >> 2. i'm free in choice where to put the implementation of this<br>
>> >> function, or even build VM initially w/o this function.<br>
>> >> 3. across different platforms, one can use same VMMaker to generate<br>
>> >> sources , and it will always compile & link.<br>
>> >><br>
>> >> doesn't it look like more modular?<br>
>> ><br>
>> > No. It looks utterly pointless to me. You introduce a plugin that does<br>
>> > nothing but looking up and call an atom; what good is that plugin? If<br>
>> > you<br>
>> > generalize that just a little you have the FFI where you might declare<br>
>> > ioFoo() directly and call it. Which of course could be done via atom<br>
>> > table<br>
>> > too, but I still fail to see how that would be more modular.<br>
>> ><br>
>><br>
>> Not a plugin. I mentioned VM code in interp.c. I see little need in<br>
>> having such constructs in plugin.<br>
>> In VM core, however, there is always a bit of uncertainty - VM forced<br>
>> to provide primitive by default (because its a standart one), but on<br>
>> different platforms, depending on their capabilities or your intent,<br>
>> you may omit putting functionality of some primitives into VM.<br>
>> This is where such scheme is can be quite useful - instead of stubbing<br>
>> function in sources, or exploring what function has to return to make<br>
>> primitive fail , just remove it from build.<br>
>><br>
>> >> Now, take a look at a sqWin32Stubs.c - how it would look if we would<br>
>> >> use shared namespace? It would look as zero length file.<br>
>> >> Because all we have to do in platform code is just initialize pointers:<br>
>> >><br>
>> >> pointers = {<br>
>> >> "ioFoo" , ioFoo<br>
>> >> #ifdef NO_SOME_STUFF<br>
>> >> "ioBar" , ioBar<br>
>> >> #endif<br>
>> >> ...<br>
>> >> };<br>
>> ><br>
>> > And if you stick this in sqWin32Stubs.c (or its equivalent) you end up<br>
>> > with<br>
>> > a non-empty stubs file. In other words, you are replacing one set of<br>
>> > stubs<br>
>> > with another one. Not much of an improvement.<br>
>> ><br>
>><br>
>> Its not the same thing. Currently you have to define a function - with<br>
>> implementation or with empty body to make compiler content.<br>
>> In example i showed - you can do simpler - just omit binding a<br>
>> function to a symbol.<br>
>> Suppose we having a well structured organization of VM platform<br>
>> sources, then it might look like:<br>
>><br>
>> #ifdef SUPPORT_FILES<br>
>> #include "file_functions.inc"<br>
>> /// or put all functions here w/o using #include<br>
>> #endif<br>
>><br>
>> instead of something like:<br>
>><br>
>> #ifdef SUPPORT_FILES<br>
>> #include "file_functions.inc"<br>
>> /// or put all functions here w/o using #include<br>
>> #else<br>
>> #include "file_functions_stubs.inc"<br>
>> #endif<br>
>><br>
>> >> I'm currently trying to make a HostWindowsPlugin and want to put all<br>
>> >> windowing and i/o stuff in it from VM platform sources.<br>
>> ><br>
>> > Why do another one? Is there something that the current host window<br>
>> > plugin<br>
>> > doesn't address?<br>
>><br>
>> Well, there's many things.<br>
>> Don't want to OT here, just simple example: when image fires up, i<br>
>> want to show a splash screen, and then, when user clicks on it, or<br>
>> some time passed , hide it and show the "main" window. ALL is<br>
>> controlled from image, of course , e.g. one might want to show splash<br>
>> screen and when user clicks on it - just quit the squeak :)<br>
>><br>
>> ><br>
>> >> It would be much a cut'n'paste experience to me, if all callouts in VM<br>
>> >> would use atoms - because then i don't need to touch headers & sources<br>
>> >> and many different places to make compiler content. Because there<br>
>> >> would be no need in exporting function in C-like manner.<br>
>> ><br>
>> > Ah. But that's a fallacy. That you don't need to "touch" these places<br>
>> > doesn't mean you don't need to know about them. In fact, having to touch<br>
>> > them, having the compiler complain about them is a great way to learn<br>
>> > and<br>
>> > know and understand all the areas you need to touch - if the VM would<br>
>> > randomly crash on you because you have replaced one but not another<br>
>> > function<br>
>> > you'd be in for a hellish experience. Yes, the C compiler can be<br>
>> > notorious<br>
>> > but it can also be a pretty good teacher.<br>
>> ><br>
>><br>
>> The same self-fallacy is to be confident, that if your code compiles<br>
>> ok it doesn't crash in some random place :)<br>
>><br>
>> >> And lastly, remember InterpreterProxy thing? Each time you want to<br>
>> >> introduce new functionality , you have to extend the structure and<br>
>> >> increase version number. But what is it? Its just a list of pointers!<br>
>> ><br>
>> > No, it's not. It's an interface (a contract) between the VM and the<br>
>> > plugin.<br>
>> ><br>
>> >> Isn't it would be simpler for plugin to just lookup a function by its<br>
>> >> name - and use it if such function exists. Its pretty much same as<br>
>> >> dynamic linking, just s/dlsym/getAtomValue/ and don't let your code be<br>
>> >> constrained by compiler/linker anymore :)<br>
>> ><br>
>> > I *very much* doubt that it would be simpler for each plugin to look up<br>
>> > the<br>
>> > function every time they use it and test for its existence every time it<br>
>> > is<br>
>> > used. Consider this primitive:<br>
>> ><br>
>> > primitiveStringClone<br>
>> > interpreterProxy methodArgument = 1<br>
>> > ifFalse:[^interpreterProxy primitiveFail].<br>
>> > arg := interpreterProxy stackValue: 0.<br>
>> > (interpreterProxy isBytes: arg)<br>
>> > ifFalse:[^interpreterProxy primitiveFail].<br>
>> > clone := interpreterProxy clone: arg.<br>
>> > interpreterProxy pop: 2.<br>
>> > interpreterProxy push: clone.<br>
>> ><br>
>> > Now let's rewrite this in pseudo-code to see what it would look like<br>
>> > without<br>
>> > an interface:<br>
>> ><br>
>> > primitiveStringClone<br>
>> > ((interpreterProxy has: #methodArgumentCount)<br>
>> > and:[interpreterProxy methodArgument = 1])<br>
>> > ifFalse:[^interpreterProxy primitiveFail].<br>
>> > (interpreterProxy has: #stackValue)<br>
>> > ifFalse:[^interpreterProxy primitiveFail].<br>
>> > arg := interpreterProxy stackValue: 0.<br>
>> > (interpreterProxy has: #isBytes)<br>
>> > ifFalse:[^interpreterProxy primitiveFail].<br>
>> > (interpreterProxy isBytes: arg)<br>
>> > ifFalse:[^interpreterProxy primitiveFail].<br>
>> > (interpreterProxy has: #clone)<br>
>> > ifFalse:[^interpreterProxy primitiveFail].<br>
>> > clone := interpreterProxy clone: arg.<br>
>> > (interpreterProxy has: #pop)<br>
>> > ifFalse:[^interpreterProxy primitiveFail].<br>
>> > interpreterProxy pop: 2.<br>
>> > (interpreterProxy has: #push)<br>
>> > ifFalse:[^interpreterProxy primitiveFail].<br>
>> > interpreterProxy push: clone.<br>
>> ><br>
>> ><br>
>> > Simpler? You've got to be kidding me ;-)<br>
>> ><br>
>><br>
>> This counter-example missing a point :)<br>
>> Take a look at Hydra code, where InterpreterProxy has left with only 3<br>
>> functions, and will stay to have 3 functions forever without need in<br>
>> extending protocol , because rest functions is obtained dynamicaly<br>
>> using name lookup.<br>
>> A plugin simply refuse to start without having all requested VM<br>
>> functions be non-null. But the trick is, that VM not forced anymore to<br>
>> support obsolete stuff of ever-growing InterpreterProxy protocol.<br>
><br>
> I like this very much. Not totally, just very much :)<br>
> What's good here is that the plugin can access functions directly in the VM<br>
> and not go through the slow interpreterProxy. Because C supports syntax for<br>
> calls through function pointers that looks just like normal functions calls<br>
> wouldn't look very different to normal calls. The function pointers simply<br>
> need to be decared.<br>
<br>
</div></div>right!<br>
<div class="Ih2E3d">> But what I don't like is having interpreterProxy persist at all. Why not get<br>
> rid of it except for initialization and pass it in through setInterpreter?<br>
> What do you need interpreterProxy to persist for?<br>
</div>right!<br>
<div class="Ih2E3d">> So Slang generates a standard prolog for all the functions used in a plugin<br>
> at the beginning of the file (nice documentation; lists exactly the<br>
> functions the plugin uses). setInterpreter (better called<br>
> initializePlugin?) takes an argument that provides a function to lookup<br>
> function pointers by name. Slang generates the code to initialize these<br>
> fnction pointers. Uses of the function pointers look just like normal<br>
> calls.<br>
</div>Right!<br>
<div class="Ih2E3d">> Then internal plugins can be linked directly against the VM. e.g.<br>
><br>
<br>
</div>Things works exactly in that way in Hydra. :)<br>
<div class="Ih2E3d"><br>
> #if EXTERNAL_PLUGIN<br>
> static void (*popthenPush(sqInt,sqInt);<br>
> static void (*primitiveFailed)(void);<br>
> #endif<br>
> ...<br>
> void somePrimitive()<br>
> {<br>
> ....<br>
> if (!ok) {<br>
> primitiveFailed();<br>
> return;<br>
> }<br>
> popthenPush(result);<br>
> }<br>
> ...<br>
> #if EXTERNAL_PLUGIN<br>
> static sqInt<br>
> setInterpreter(InterpreterProxy *interpreterProxy)<br>
> {<br>
> if (interpreterProxy.oopSize() != 4)<br>
> return InitErrorWrongOopSize;<br>
> if (!interpreterProxy.getFunction("popthenPush", &popthenPush)<br>
> || !interpreterProxy.getFunction("primitiveFailed", & primitiveFailed))<br>
> return InitErrorMissingFunction;<br>
> return 0;<br>
> }<br>
> #endif /* EXTERNAL_PLUGIN */<br>
>><br>
>><br>
>> > Cheers,<br>
>> > - Andreas<br>
>> ><br>
>><br>
>><br>
>> --<br>
>> Best regards,<br>
>> Igor Stasenko AKA sig.<br>
>><br>
><br>
><br>
><br>
><br>
><br>
<br>
<br>
<br>
</div>--<br>
<div><div></div><div class="Wj3C7c">Best regards,<br>
Igor Stasenko AKA sig.<br>
<br>
</div></div></blockquote></div><br></div>