[Vm-dev] [Vm-beginners] Stability of the external plugin interface

[Eliot Miranda]

Hi Tim,

> On May 25, 2020, at 1:50 PM, tim Rowledge <tim at rowledge.org> wrote:
> 
> 
>> On 2020-05-25, at 10:25 AM, K K Subbu <kksubbu.ml at gmail.com> wrote:
>> 
>>> On 25/05/20 5:04 pm, Jakob Reschke wrote:
>>> If a library breaks compatibility, you would have to update the plugin and maybe think about how to stay compatible with older Squeaks. If it were FFI with a proper facade that finds out how to link to the external library, you would need to update the facade and worry there about staying compatible, but you could do it in Smalltalk. It requires discipline and guidance to only use the facade, of course.
>> Libraries like OpenGL are frustrating because of poorly abstracted, low level functions. Trying to stable facade around its warts will not solve the core problem of having to deal with static code and compile cycles.
>> 
>> The frustration is having to regress from live programming into code-compile-link-test cycle in 2020! Given a choice between live programming and backward compatibility, I would choose live programming because I can close any gaps through live coding.
> 
> 
> Nicely put.
>> 
> 
>> For instance, a few years back I developed live previews for LaTeX without leaving Squeak (Etoys) by using OSProcess to build a LaTeX pipeline that converted code into image as the code was being typed. It is an inefficient design but it took just half a day and has served well over the years.
>> 
>> But this was at process level. If only I could do the same with DLLs. If only Squeak had a tool (ELF reader?) to reify a DLL, extract its instruction set, interface definitions, state variables and calling conventions. Then I can livecode a class and dynamically compile it (now that we have JIT) to call functions DLL without leaving the image. The code may not be 'portable' in traditional sense. But it will be a lot more manageable than static designs.
> 
> 
> VW has a parser of C header files that is I suppose the right way to do this sort of thing - you parse the headers (warning - I seem to recall parsing 'windows.h' taking several days in the original VW ddl product) and build proxies of all the types/structs/etc.  Is that a practical way to do it? Hell of a lot of work to do, certainly.

Emphatically not.  DLLCC’s C parser is a nightmare to maintain, and with a little cunning, essentially unnecessary. See all the conversation around generating a C program which when run outputs information that the image can easily parse, and as a result (re)populate a shared pool with relevant platform specifics.  We have this in FFISharedPool (by Shaping IIRC, after an idea/suggestion of mine, implemented as a prototype before I left Cincom, and as a part of UFFI by Mariano Martinez Peck, following discussion a few years back).

Generating a program has several advantages

- no need to implement any part of the C toolchain (doesn’t reinvent the wheel)

- uses the platform’s information more-or-less directly, rather than trying to mimic it

- the binaries can be shared between developers (a library of them?) and deployed with cross-platform apps

- only the FFI api author needs a C development toolset


So what the FFI author does is define a shared pool which has platform-specific metadata, specifically the necessary include files and version selection defines for each relevant platform.  The FFI shared pool machinery takes care of generating the program when necessary (only when authoring an FFI), and running it when necessary (only once for a single-platform app, only when launching on a different platform to the one snapshotted on)


> 
> tim
> --
> tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
> Strange OpCodes: QVC: Question Valid Command
> 
>