[squeak-dev] Loading FFI is broken

Eliot Miranda eliot.miranda at gmail.com
Fri Nov 15 01:40:11 UTC 2013


Hi David,

    forgive me for chipping in somewhat tangentally.  But the vision thang
can be important sometimes.


On Thu, Nov 14, 2013 at 3:59 PM, David T. Lewis <lewis at mail.msen.com> wrote:

> A few things to be aware of:
>
> - Both Eliot Miranda and Igor Stasenko have done considerable work on
> FFI interfaces. I cannot explain the details, but be aware that there
> may be more than one kind of FFI that people will want to use, and it
> seems quite likely that the classic FFI may in future be replaced by
> a different implementation.
>

+1.  At least Spur will change the facilities available as does
NativeBoost.  Ideally we'll develop some kind of synthesis.  SPur provides
pinning.  NativeBoost provides the ability to generate marshalling code
from within the image which means not having to rely on the VM for
marshalling, which should hopefully mean correctness on platforms with
really complex marshalling rules such as x86-64/IA64.

Further, Igor's discovery that any C signature can be compiled as a
Smalltalk literal array changes how one should parse C declarations.  It's
a magnificently simple hack.

        struct { int foo; float bar; } (*f)(double bletch);
=>  #(#struct #'{' #int #foo #';' #float #bar  #';' #'}' #(#* #f) #(#double
#bletch) #';')
=>  #(struct { int foo; float bar; } (*f)(double bletch);)


- None of the available FFI interfaces will work for a VM compiled in
> 64-bit mode, nor can they be used to interface to existing 64-bit
> libraries.
> I am sure that the problems will be addressed in one or more of the
> FFI implementations, but this may take some time (to give an idea of
> the time scale, see http://bugs.squeak.org/view.php?id=7237).
>

That will be addressed when a 64-bit Spur is implemented.


> - Like any other reloadable package in Squeak, the classic Squeak FFI
> can be maintained as an external package. It requires a little more
> work to keep it healthy, because someone has to remember to test it
> once in a while. But there is no reason that this can not or should not
> be done.
>

+1.

Dave
>
> On Thu, Nov 14, 2013 at 03:38:49PM -0800, Chris Cunningham wrote:
> > FFI used to BE part of the base image, and was specifically taken out on
> > purpose.  The purpose, as I remember it, was because it acted like a
> > security hole for squeak as a squeaklet on the web (when you browse a
> page
> > on a website that invokes Squeak, and then being able to, via FFI, invoke
> > anything you want on the remote machine).  So, for any image still
> working
> > in that mode (such as the etoys derivative), it is important to leave
> out.
> >
> > But, with the shiny new CI servers, couldn't we have an artifact built
> with
> > FFI loaded?
> >
> > Or, even neater, include FFI in the trunk, but have a switch/pragma in
> the
> > image that says 'Don't ever load FFI' for those that don't want it, but
> > want everything else loaded?  So, for people that LIKE FFI, it is just
> part
> > of Trunk; for those that have to avoid it, it isn't loaded; for the rest,
> > they get whatever was setup when they got their image.  (I'd be in the
> > first group most of the time)
> >
> > -cbc
> >
> >
> > On Thu, Nov 14, 2013 at 2:30 PM, Frank Shearar <frank.shearar at gmail.com
> >wrote:
> >
> > > On 14 November 2013 16:42, Eliot Miranda <eliot.miranda at gmail.com>
> wrote:
> > > > Hi Frank,
> > > >
> > > >
> > > > On Thu, Nov 14, 2013 at 1:59 AM, Frank Shearar <
> frank.shearar at gmail.com>
> > > > wrote:
> > > >>
> > > >> That's the source of the problem: FFI-Kernel uses FFIConstants,
> which
> > > >> is declared/defined in FFI-Pools.
> > > >>
> > > >> Insert standard Frank rant of something called "Kernel" depending on
> > > >> other stuff. (Possibly insert standard Chris rant of a package
> > > >> containing a single class? :) )
> > > >
> > > >
> > > > The problem here is that the VM depends on FFIConstants also, and
> the VM
> > > > shouldn't depend on the rest of FFI.  So FFIConstants does need to
> be on
> > > its
> > > > own, but could perhaps be called something different.
> > >
> > > You know, I would be quite happy if we folded FFI into the base image.
> > > Seriously. For once I'd argue for _including_ something. I would
> > > gladly trade Morphic for FFI.
> > >
> > > OK, I'll chalk this up to That's How It Is, and move on. If you
> > > install from SqueakMap (as you should, and as I should have) you get
> > > the right load order.
> > >
> > > frank
> > >
> > > >> frank
> > > >>
> > > >> On 12 November 2013 23:20, Chris Muller <ma.chris.m at gmail.com>
> wrote:
> > > >> > I didn't see you loading FFI-Pools in advance.  The issue you
> > > >> > encountered
> > > >> > may not have had anything to do with -eem.24.
> > > >> >
> > > >> > FYI -- FFI also has a "head" release on SqueakMap which documents
> how
> > > to
> > > >> > load it.
> > > >> >
> > > >> >
> > > >> > On Tue, Nov 12, 2013 at 4:50 PM, Frank Shearar <
> > > frank.shearar at gmail.com>
> > > >> > wrote:
> > > >> >>
> > > >> >> That pulls in the latest packages, right?
> > > >> >>
> > > >> >> So that does work. But why does the reported version fail to
> load?
> > > >> >> That's a problem still.
> > > >> >>
> > > >> >> (While this lets me get on with what I wanted to do - add
> > > >> >> #interleaving: to Xtreams - this is still a problem. But thanks
> for
> > > >> >> the workaround, Chris!)
> > > >> >>
> > > >> >> frank
> > > >> >>
> > > >> >> On 12 November 2013 20:55, Chris Muller <asqueaker at gmail.com>
> wrote:
> > > >> >> > Installer new merge: #ffi.
> > > >> >> >
> > > >> >> > Worked for me..
> > > >> >> >
> > > >> >> > On Tue, Nov 12, 2013 at 2:45 PM, Frank Shearar
> > > >> >> > <frank.shearar at gmail.com>
> > > >> >> > wrote:
> > > >> >> >> On 26 February 2013 18:36, Eliot Miranda <
> eliot.miranda at gmail.com
> > > >
> > > >> >> >> wrote:
> > > >> >> >>>
> > > >> >> >>>
> > > >> >> >>> On Tue, Feb 26, 2013 at 1:51 AM, Frank Shearar
> > > >> >> >>> <frank.shearar at gmail.com>
> > > >> >> >>> wrote:
> > > >> >> >>>>
> > > >> >> >>>> (Installer monticello mc: (MCHttpRepository new location:
> > > >> >> >>>> 'http://source.squeak.org/FFI'))
> > > >> >> >>>>     install: 'FFI-Kernel-eem.24.mcz'. "There's a -tbn.25,
> but
> > > >> >> >>>> that's
> > > >> >> >>>> not important to this post"
> > > >> >> >>>>
> > > >> >> >>>> fails with an MNU: ExternalFunction class >>
> > > >> >> >>>> callingConventionFor:.
> > > >> >> >>>>
> > > >> >> >>>> As far as I can see what's happening is this:
> > > >> >> >>>> * during the loading of the mcz ExternalFunction is defined,
> > > >> >> >>>> * a method is parsed (#XOpenDisplay, which has a pragma
> <cdecl:
> > > >> >> >>>> X11Display* ''XOpenDisplay'' (char*) module:''X11''>)
> > > >> >> >>>> * Parser >> externalFunctionDeclaration checks whether
> > > >> >> >>>> ExternalFunction is defined.
> > > >> >> >>>> * It is, so tries to evaluate `descriptorClass
> > > callingConvention:
> > > >> >> >>>> here` and boom, because ExternalFunction class >>
> > > >> >> >>>> callingConventionFor: _has not been loaded yet_.
> > > >> >> >>>
> > > >> >> >>>
> > > >> >> >>> I thought we'd modified Monticello to load new methods first.
> > >  Did
> > > >> >> >>> this not
> > > >> >> >>> get added to trunk?
> > > >> >> >>
> > > >> >> >> Apparently not. It still happens with an up-to-date Squeak
> 4.5.
> > > >> >> >>
> > > >> >> >> frank
> > > >> >> >>
> > > >> >> >>>> I've seen this kind of issue with Helvetia code: sometimes
> you
> > > >> >> >>>> simply
> > > >> >> >>>> have to load class-side methods first.
> > > >> >> >>>>
> > > >> >> >>>> Thoughts? Lukas worked around the issue with his Helvetia
> code
> > > by
> > > >> >> >>>> directly patching Monticello, and ripping out its "try to do
> > > >> >> >>>> atomic
> > > >> >> >>>> loading" mechanism.
> > > >> >> >>>>
> > > >> >> >>>> frank
> > > >> >> >>>>
> > > >> >> >>>
> > > >> >> >>>
> > > >> >> >>>
> > > >> >> >>> --
> > > >> >> >>> best,
> > > >> >> >>> Eliot
> > > >> >> >>>
> > > >> >> >>>
> > > >> >> >>>
> > > >> >> >>
> > > >> >> >
> > > >> >
> > > >> >
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > best,
> > > > Eliot
> > >
> > >
>
> >
>
>
>


-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20131114/89a88985/attachment.htm


More information about the Squeak-dev mailing list