[Vm-dev] Re: Alien and FFI integration

[Eliot Miranda]

Hi Andreas,
    what I really think about this is that call-outs are the weakest part of
Alien.  The callout facilities are essentially a hack that works on IA32,
could work on ARM or PowerPC at a pinch, and will fail horribly in general
on x86-64 (um EMT64 & AMD64).  e.g. the AMD64 calling convention splits
struct contents across available registers as needed so e.g. a struct with
one float, four longs and a float will get passed in fp arg reg 0 & fp arg
reg 1 (for first and last fields) and in int arg regs 0 through 3 (for the
second through the fifth fields).

Yes, one can ask the programmer to work this out and use a low level
interface but one will get rightly a rude answer to the question.  But the
only  real answer is to keep call specs and have the image compile them into
something that matches the ABI.

For a while when David Simmons and I have discussed this we have agreed that
the right way to do it is have an ABI compiler framework in the image whose
job it is to compile code to make the callout on first use.  When we've
discussed it we've suggested that the ABI compiler produces some kind of RTL
language that the JOT consumes and turns into proper machine code to
actually make the callout.  But as you noted on Monday, the ABI compiler
could actually produce machine code.  Alien already does this for producing
the callback thunks for callback support.  The only issues here are how to
plumb in the machine code to the machine code the JIT generates for methods
and (for me) how to avoid the ABI compiler mushrooming into a full blown
native compiler with too much duplication.

On Sun, Mar 1, 2009 at 11:56 PM, Andreas Raab <andreas.raab at qwaq.com> wrote:

> Hi Eliot -
>
> [cc: vm-dev to see if anyone has some good ideas] I was thinking about how
> one would move between FFI and Alien more freely and one thing that seemed
> interesting is to have the compiler simply generate Alien call code for FFI
> specs. E.g., when the compiler encounters, say:
>
> system: aString
>    <apicall: ulong 'system' (char*) module: 'libc'>
>    ^self externalCallFailed
>
> it would generate something like here:
>
> system: aString
>    AlienCall ifNil:[
>        "Register this method to be flushed on shutdown"
>        Alien registerMethod: thisContext method.
>        "Lookup the call if it isn't cached"
>        AlienCall := Alien lookup: 'system' inLibrary: 'libc'.
>        "Result can use a cached Alien since it never changes"
>        AlienResult := Alien new: 4.
>    ].
>    "Arg1 needs to be reloaded every time we call"
>    Arg1 := Alien fromString: aString.
>    AlienCall primFFICallResult: AlienResult with: Arg1.
>    Arg1 free. "or however one frees those"
>    "XXXX: Would need some success indication here"
>    success ifFalse:[^AlienResult asUnsignedLong].
>
>    "Non-primitive code follows here"
>    ^self externalCallFailed "aka: (thisContext sender method literalAt: 1)
> reportLastError"
>
> The "globals" in the above (AlienCall, AlienResult, AlienArg1) are all
> method literals to cache the information (have you ever considered adding a
> <static: foo> directive to the compiler to mark temp foo as a temp stored in
> the literal frame?).
>
> There are two advantages and a major disadvantage to this approach. The
> good news is that this would allow one to continue to use existing FFI specs
> with Alien (i.e., ease migration). Plus, some optimizations (like caching
> lookups and args) can be done by the compiler now and don't require people
> to write code themselves.
>
> The obvious disadvantage is that you need to change the code to adjust for
> changes in the ABI. Which sucks. I really wish Alien could take a descriptor
> and deal with the ABI details itself. I wonder if it's feasible to have
> subclasses of Alien function calls that implement the different ABIs and
> pass enough information along that the ABI dependent parts could be done
> there?
>
> Cheers,
>  - Andreas
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20090303/c8bf7072/attachment.htm