[Vm-dev] Re: [Pharo-dev] NativeBoost

Esteban Lorenzano estebanlm at gmail.com
Wed Nov 26 07:30:52 UTC 2014


> On 25 Nov 2014, at 20:27, Markus Fritsche <mfritsche at reauktion.de> wrote:
> 
> 
> 
> On Tue, Nov 25, 2014 at 5:06 AM, Torsten Bergmann <astares at gmx.de <mailto:astares at gmx.de>> wrote:
> Please enlighten me as well. I also do not yet understand the idea of this unification layer.
> 
> "Unification layer" is just a name.  IMO Ronie is doing something much more useful.  He is architecting the marshalling layer of the FFI so that it can 
> - be portable
> - work in both the Cog JIT and the Interpreter
> This is effectively an abstract instruction set for executing low-level machine intsructions.  It can be used for FFI call-out marshalling, but it can also be used for low-level code generation, and we are using it in Sista.  COnsequently Ronie's "Low code" as it is properly called can be an integral component to both a high-quality FFI and a fast VM.
> 
> For me this is a useful substrate for an FFI that can be high quality.  But note that it is only one component.  The other components are
> - an image-level ABI compiler whose job it is to generate the correct marshalling code for different platforms.  The FFI for processors such as IA64/x86-64 is complex, and a compiler is the only performant way to generate correct marshalling code
> - callback machinery.  I have already designed and implemented this in the context of Alien.  The architecture is portable; it can function correctly on x86-64 and ARM, not just x86.
> 
> 
> So far we have
>  1. FFI   -> not maintained very well, no callbacks
> 
> This is simply false.  The FFI was in active use at Qwaq where Cog was first implemented.  I have continued to maintain and enhance it, reimplementing it so that it is a pure Smalltalk plugin with no assembler support code, so that it is reentrant.  Recently Doug McPherson has implemented the ARM version alongside my original x86 version.

yep, I’m aware and I was trying to have time to bring those changes into Pharo. Hopefully soon.  

>  
>  2. Alien -> should provide the callbacks, hard to find a predefined package/VM combination
> 
> Every Cog VM supports Alien callbacks.  The Alien package Alien-eem.24 at http://www.squeaksource.com/Alien <http://www.squeaksource.com/Alien> "jsut works", at least in Squeak.  Try it.


it works. 
We have ConfigurationOfOldAlien that loads all required (Is a bad name I choose because there was your sources and a fork from Luc and there were not in sync, so I took your sources and made a Configuration for them :P)

>  
>  3. NativeBoost was supposed to be the better one, was based on AsmJit and replace 1. and 2.
> 
> I disagree.  NativeBoost has not been designed with portability in mind, nor has it been designed with interpretive VMs in mind.  Further Igor made no attempt to work with me in providing the support he needs from Cog to integrate NativeBoost when I visited Rmod early this year.
> 
> 
> According to my knowledge it was the goal to continue with NB, integrate it (which is done)
> and move it forward as the single solution for external calls and callbacks.
> 
> So why do we need another layer (and mixed approach with still old FFI) instead of working
> on bringing NB to the other platforms as well and work on a good NativeBoost. Wasn't there
> already work for AsmJit for ARM?
> 
> We have a functional FFI at the moment, FFI + Alien for callbacks.  This is in industrial use, both at Terf (nee Qwaq) and Cadence.  We have NativeBoost that does not work on ARM.  We have Ronie's work on low code that fits both with a well-architected FFI and with Sista.
> 
> I am frustrated that there is no coherence in our work here.  I have a clear understanding of what architecture can work, a clear vision, and yet other than Ronie, all I see is FUD ("Alien doesn't work", "FFI doesn't work", "Alien callbacks don't work"; all false).  I wish this wasn't the case.  I do not have time right now to work on all of Spur, Spur 64, Sista and find time to architect the FFI.  But I know how this stuff works (I've implemented what works now) and as the leads VM architect isn't it right that the community try and work with me rather than without me?

Honestly, I’m quite frustrated too. 
And I cannot agree with you more. 
But hopefully we will start to work on this problem and remove all discordances soon (I know in all other VM aspects we already did it and now we work aligned, so we can do this for FFI too… in fact last month I wanted to work on it but then other work became more urgent). 

Esteban

> 
> 
> Thx
> T.
> 
> > Gesendet: Dienstag, 25. November 2014 um 10:01 Uhr
> > Von: "Markus Fritsche" <mfritsche at reauktion.de <mailto:mfritsche at reauktion.de>>
> > An: "Pharo Development List" <pharo-dev at lists.pharo.org <mailto:pharo-dev at lists.pharo.org>>
> > Betreff: Re: [Pharo-dev] NativeBoost
> >
> > On 2014-11-24 23:34, Esteban Lorenzano wrote:
> >
> > > now, since the most common use of NB is for FFI, and NB is not present
> > > in all platforms, we are working (Ronie, in this case) in a “unified
> > > FFI”, which will provide a common abstraction layer for several FFI
> > > frameworks (NB, OldFFI, Alien…). With that, we can choose which one
> > > of the frameworks we use depending on the situation (but ideally, we
> > > will maintain just one, the one that fits better in Pharo).
> >
> > I feel that providing a common abstraction layer on top of the various
> > approaches to interface to C/ C++ libraries won't be useful and should
> > be considered a waste of resources. From what I understood from the
> > documentation, NativeBoost seems to be the most complete interface
> > technology yet. I would rather (as in: If I was capable) try to get
> > NativeBoost into Squeak and add the functionality missing that still
> > justifies the existence of FFI and Alien than creating an abstraction
> > layer that can't cope with the differences in the tools (which was the
> > reason for their first creation despite the available ones).
> >
> > I tried to play around with FFI and creating plugins in Squeak and
> > failed (I didn't try very hard though). NativeBoost was the first thing
> > I was able to wrap a DLL myself and understand how to do it. The only
> > thing that I wasn't able to wrap my mind around was finalization, but
> > that is also due to the fact that I couldn't find understandable (read:
> > understandable to me) documentation about the underlying concept in
> > Pharo.
> >
> > (OT: Finalization and concurrency probably is more about the concept
> > rather than the implementation in Pharo?)
> >
> > This is a piece of opinion. We all know, that unfortunately, one doesn't
> > have to have knowledge to have an opinion, so if my opinion is wrong due
> > to lack of knowledge, please ignore it.
> >
> > Best regards,
> >    Markus
> >
> >
> >
> 
> 
> 
> 
> -- 
> best,
> Eliot

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20141126/19d229cc/attachment.htm


More information about the Vm-dev mailing list