[Vm-dev] Re: [Pharo-dev] NativeBoost

Tue Nov 25 18:37:34 UTC 2014

On Tue, Nov 25, 2014 at 5:06 AM, Torsten Bergmann <astares at gmx.de> wrote:

> Please enlighten me as well. I also do not yet understand the idea of this
> unification layer.
>

"Unification layer" is just a name.  IMO Ronie is doing something much more
useful.  He is architecting the marshalling layer of the FFI so that it can
- be portable
- work in both the Cog JIT and the Interpreter
This is effectively an abstract instruction set for executing low-level
machine intsructions.  It can be used for FFI call-out marshalling, but it
can also be used for low-level code generation, and we are using it in
Sista.  COnsequently Ronie's "Low code" as it is properly called can be an
integral component to both a high-quality FFI and a fast VM.

For me this is a useful substrate for an FFI that can be high quality.  But
note that it is only one component.  The other components are
- an image-level ABI compiler whose job it is to generate the correct
marshalling code for different platforms.  The FFI for processors such as
IA64/x86-64 is complex, and a compiler is the only performant way to
generate correct marshalling code
- callback machinery.  I have already designed and implemented this in the
context of Alien.  The architecture is portable; it can function correctly
on x86-64 and ARM, not just x86.

> So far we have
>  1. FFI   -> not maintained very well, no callbacks
>

This is simply false.  The FFI was in active use at Qwaq where Cog was
first implemented.  I have continued to maintain and enhance it,
reimplementing it so that it is a pure Smalltalk plugin with no assembler
support code, so that it is reentrant.  Recently Doug McPherson has
implemented the ARM version alongside my original x86 version.

>  2. Alien -> should provide the callbacks, hard to find a predefined
> package/VM combination
>

Every Cog VM supports Alien callbacks.  The Alien package Alien-eem.24 at
http://www.squeaksource.com/Alien "jsut works", at least in Squeak.  Try it.

>  3. NativeBoost was supposed to be the better one, was based on AsmJit and
> replace 1. and 2.
>

I disagree.  NativeBoost has not been designed with portability in mind,
nor has it been designed with interpretive VMs in mind.  Further Igor made
no attempt to work with me in providing the support he needs from Cog to
integrate NativeBoost when I visited Rmod early this year.

According to my knowledge it was the goal to continue with NB, integrate it
> (which is done)
> and move it forward as the single solution for external calls and
> callbacks.
>
> So why do we need another layer (and mixed approach with still old FFI)
> instead of working
> on bringing NB to the other platforms as well and work on a good
> NativeBoost. Wasn't there
> already work for AsmJit for ARM?
>

We have a functional FFI at the moment, FFI + Alien for callbacks.  This is
in industrial use, both at Terf (nee Qwaq) and Cadence.  We have
NativeBoost that does not work on ARM.  We have Ronie's work on low code
that fits both with a well-architected FFI and with Sista.

I am frustrated that there is no coherence in our work here.  I have a
clear understanding of what architecture can work, a clear vision, and yet
other than Ronie, all I see is FUD ("Alien doesn't work", "FFI doesn't
work", "Alien callbacks don't work"; all false).  I wish this wasn't the
case.  I do not have time right now to work on all of Spur, Spur 64, Sista
and find time to architect the FFI.  But I know how this stuff works (I've
implemented what works now) and as the leads VM architect isn't it right
that the community try and work with me rather than without me?

> Thx
> T.
>
> > Gesendet: Dienstag, 25. November 2014 um 10:01 Uhr
> > Von: "Markus Fritsche" <mfritsche at reauktion.de>
> > An: "Pharo Development List" <pharo-dev at lists.pharo.org>
> > Betreff: Re: [Pharo-dev] NativeBoost
> >
> > On 2014-11-24 23:34, Esteban Lorenzano wrote:
> >
> > > now, since the most common use of NB is for FFI, and NB is not present
> > > in all platforms, we are working (Ronie, in this case) in a “unified
> > > FFI”, which will provide a common abstraction layer for several FFI
> > > frameworks (NB, OldFFI, Alien…). With that, we can choose which one
> > > of the frameworks we use depending on the situation (but ideally, we
> > > will maintain just one, the one that fits better in Pharo).
> >
> > I feel that providing a common abstraction layer on top of the various
> > approaches to interface to C/ C++ libraries won't be useful and should
> > be considered a waste of resources. From what I understood from the
> > documentation, NativeBoost seems to be the most complete interface
> > technology yet. I would rather (as in: If I was capable) try to get
> > NativeBoost into Squeak and add the functionality missing that still
> > justifies the existence of FFI and Alien than creating an abstraction
> > layer that can't cope with the differences in the tools (which was the
> > reason for their first creation despite the available ones).
> >
> > I tried to play around with FFI and creating plugins in Squeak and
> > failed (I didn't try very hard though). NativeBoost was the first thing
> > I was able to wrap a DLL myself and understand how to do it. The only
> > thing that I wasn't able to wrap my mind around was finalization, but
> > that is also due to the fact that I couldn't find understandable (read:
> > understandable to me) documentation about the underlying concept in
> > Pharo.
> >
> > (OT: Finalization and concurrency probably is more about the concept
> > rather than the implementation in Pharo?)
> >
> > This is a piece of opinion. We all know, that unfortunately, one doesn't
> > have to have knowledge to have an opinion, so if my opinion is wrong due
> > to lack of knowledge, please ignore it.
> >
> > Best regards,
> >    Markus
> >
> >
> >
>
>

-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20141125/1826da92/attachment.htm