[Vm-dev] [Vm-beginners] Stability of the external plugin interface

[tim Rowledge]

> On 2020-05-24, at 3:57 PM, Eliot Miranda <eliot.miranda at gmail.com> wrote:
> 
> Plugins emerged from the lack of an FFI, later a deficient FFI (slow, non-reentrant, no callbacks), and the need for security, at a time when graphics co-processing was very rare and not at all commodity technology.  The world looks different now.  We should evolve to match the new environment.  I agree that security is very important.  But I do not believe that security implies no FFI. I think we can have both security and the flexibility, development speed and architectural efficiency of an FFI-based implementation.


The VM plugins were an attempt to get away from a few problems that plagued VM development at the end of the last millennium (and I don't mean March this year) with things like a limited number of prim table entires, the lack of flexibility a static table engendered, the time it took to compile an entire VM for a tiny change/test cycle etc (IIRC of the order of half an hour back then). The original external plugins stuff meant you could get to outside code via a name system that didn't screw the prim table (and I'd still love to find away to get rid of it entirely, but dream on)  and that let you build/load/try/unload/rebuild quickly. We were also starting the beginnings of the ffi stuff around then. Andy Greenberg came up with the SmartSyntaxPlugin stuff, for example, at the end of '99. Apparently I sent Dan/Andreas a final pluginized linux vm late April 2000.

It's worth remembering just how awful Windows & Mac OS were back then as part of the wider scenario - non-preemptive processscheduling, weird callback based event reading, general nonsense. Checking ancient emails to find dates etc reminded me of a huge list of other stuff we were working on back then that *still* needs finishing! And the early pinned objects stuff... and private methods! And...!) Oh-ho - and as an amusing aside, an email from Eliot of that sort of date was pointing out how amazing it was to be able to get a 30Gb disk for a mere $190; you can get 10TB now.

I definitely agree that for an awful lot of cases using a good ffi is better nowadays. It does have some costs though; but in some case I think those are costs we should be pleased to pay. For example, a number of current plugins would be better replaced with ffi classes that actually work with the OS facilities rather than trying to work around them. The FilePlugin comes readily to mind. How much nicer it would be to handle files with direct ffi calls to OS routines and have the complicated stuff in Smalltalk bridge classes. Hmm, now where did we have this in the past; oh yes, in the BrouHaHa images 30-some years ago.

Moving to ffi-only would allow removing a fair bit of cruft from the VM, though there's our old friend 'backward compatibility' to argue with. It would mean some rewriting (mostly simplification I think?) of existing plugins code. And what to do about stuff that we pretty much always compile as internal plugins? Rewrite as external libraries and pay the (hopefully small) performance cost, or rewrite as appropriate to drop any pretence of being able to be external? SoundPlugin should absolutely be replaced by a bridge class and ffi calls to the OS. LargeInterger? Probably much better to pull properly into the vm core. MiscPlugins? Wouldn't be surprised if they could all be dropped and the cog'd versions would run as fast.

Strip the VM. It should have just the bare abilities to load, run and save images, and call the OS for anything to do with i/o etc. That'll be one meeeeeeeeeeeeeeeeellion dollars please.


tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
Strange OpCodes: DCVP: Destroy another Computer Via Phone-link