[Vm-dev] Alien support in standard VM?

[Andreas Raab]

Actually, I think Michael's question was more on an immediate basis. As 
it stands there is a significant number of patches that one needs to 
scrape from the four corners of the universe before one can even begin 
to build the Alien FFI plugin. It would be helpful if this could be 
folded into VMMaker so that the plugin can be built out of the box.

Cheers,
   - Andreas

Eliot Miranda wrote:
>  
> 
> 
> ------------------------------------------------------------------------
> 
> 
> 
> On Wed, Sep 2, 2009 at 9:08 AM, Michael Haupt <mhaupt at gmail.com 
> <mailto:mhaupt at gmail.com>> wrote:
> 
> 
>     Hi,
> 
>     would it be a problem to package Alien support with the VMs by
>     default? Are there any reasons for not doing it? Can I help making
>     this happen?
> 
>     I think delivering standard images with Alien right away would be a
>     strong plus, but of course the VMs need to support that.
> 
> 
> Two things need to happen.
> 1. Alien's data manipulation facilities need to be separated from 
> Alien's very limited call facilities.
> 2. FFI's marshalling needs to be extended to marshal Aliens.
> 
> Alien's call support only works on very simple ABIs such as typical IA32 
> (x86) ones.  As soon as one confronts a machine with a register-based 
> calling convention the simple approach falls apart for many cases 
> (structure arguments, floating-point).
> 
> Further on I want to build an ABI interface compiler above the JIT that 
> can efficiently marshal arbitrary calls in arbitrary ABIs, essentially 
> reimplementing libffi.
> 
> So Michael, it would help if you would extract the data manipulation 
> parts of the IA32ABI plugin into e.g. Alienlugin and rewrite all the 
> data manipulation methods to use it.  Then shoe-horn Alien marshalling 
> into the FFIPlugin.
> 
> 
> Why reimplement libffi instead of use it directly?
> 
> First, libffi only does call-outs, not call-backs.  Alien's callback 
> architecture is quite nice; one gets a pointer to a register struct 
> (null on x86) and the stack pointer at the point of callback, wrapped up 
> in Aliens and extracts the arguments there-from.  But this needs to be 
> wrapped up in some platform-specific abstractions to present the 
> programmer with a portable interface.
> 
> Second, libffi does nothing to help translating Smalltalk objects to C 
> equivalents (mapping String instances to to null-terminated char * or 
> wchar_t * arrays, etc, etc).
> 
> Third, libffi's architecture involves building up an AST of a call 
> signature, passing the AST to a compiler which answers some marshalling 
> thunk, and then using the marshallng thunk to make a call.  The AST and 
> the marshalling thunk are C objects that one has to associate with e.g. 
> a Smalltalk compiled method and arrange that if the compiled method is 
> GCed the marshalling thunk is reclaimed.
> 
> Fourth, the marshalling thunk itself looks to be interpreted, and to 
> apply it one must call the marshaling machinery with the thunk and a 
> pointer to the vector of arguments, to have the call made.  This is a 
> little bit too much intervening machinery.
> 
> I'd rather have the JIT build the marshalling code directly into the 
> machine code generated for an external call method, translating some RTL 
> sequence that describes the marshalling operations in a 
> platform-specific manner, and write the call spec to RTL compiler in 
> Smalltalk and have it live in the image.
> 
> libffi makes a great reference for the ABI semantics, and can also be a 
> source of tests.
>  
> 
> 
>     Best,
> 
>     Michael
> 
> 
> cheers!
> Eliot 
>