[squeak-dev] Loading FFI is broken

Igor Stasenko siguctua at gmail.com
Wed Nov 20 13:42:52 UTC 2013


On 15 November 2013 02:40, Eliot Miranda <eliot.miranda at gmail.com> wrote:

> Hi David,
>
>     forgive me for chipping in somewhat tangentally.  But the vision thang
> can be important sometimes.
>
>
> On Thu, Nov 14, 2013 at 3:59 PM, David T. Lewis <lewis at mail.msen.com>wrote:
>
>> A few things to be aware of:
>>
>> - Both Eliot Miranda and Igor Stasenko have done considerable work on
>> FFI interfaces. I cannot explain the details, but be aware that there
>> may be more than one kind of FFI that people will want to use, and it
>> seems quite likely that the classic FFI may in future be replaced by
>> a different implementation.
>>
>
> +1.  At least Spur will change the facilities available as does
> NativeBoost.  Ideally we'll develop some kind of synthesis.  SPur provides
> pinning.  NativeBoost provides the ability to generate marshalling code
> from within the image which means not having to rely on the VM for
> marshalling, which should hopefully mean correctness on platforms with
> really complex marshalling rules such as x86-64/IA64.
>
> Further, Igor's discovery that any C signature can be compiled as a
> Smalltalk literal array changes how one should parse C declarations.  It's
> a magnificently simple hack.
>
>         struct { int foo; float bar; } (*f)(double bletch);
> =>  #(#struct #'{' #int #foo #';' #float #bar  #';' #'}' #(#* #f)
> #(#double #bletch) #';')
> =>  #(struct { int foo; float bar; } (*f)(double bletch);)
>
> well, actually my discovery was, that if i put
#( ... )
around any piece of code,
i can be almost certain it won't give me any syntax error's, except if
there's unmatching [] () '' thingies,
which actually NEVER occur in programming languages code - because of their
syntax.
So, my discovery was that i can use array literal syntax as a
pseudo-comment, if i like to , or as a metadata holder, as i did that in
NB..
But actually, today's NB function signature parser accepts both array
literals and strings as input. Because as it appeared, there's not much
extra code/complexity to parse strings as well.

But using the array literal for function signature has certain benefits for
developers over strings, because if i have thing in form:

#( myType foo())

i can look for senders of #myType
and quickly find where it used/defined.

While when it in form

'myType foo()'

that won't be so trivial.



> - None of the available FFI interfaces will work for a VM compiled in
>> 64-bit mode, nor can they be used to interface to existing 64-bit
>> libraries.
>> I am sure that the problems will be addressed in one or more of the
>> FFI implementations, but this may take some time (to give an idea of
>> the time scale, see http://bugs.squeak.org/view.php?id=7237).
>>
>
> That will be addressed when a 64-bit Spur is implemented.
>
>
>> - Like any other reloadable package in Squeak, the classic Squeak FFI
>> can be maintained as an external package. It requires a little more
>> work to keep it healthy, because someone has to remember to test it
>> once in a while. But there is no reason that this can not or should not
>> be done.
>>
>
> +1.
>
> Dave
>>
>> On Thu, Nov 14, 2013 at 03:38:49PM -0800, Chris Cunningham wrote:
>> > FFI used to BE part of the base image, and was specifically taken out on
>> > purpose.  The purpose, as I remember it, was because it acted like a
>> > security hole for squeak as a squeaklet on the web (when you browse a
>> page
>> > on a website that invokes Squeak, and then being able to, via FFI,
>> invoke
>> > anything you want on the remote machine).  So, for any image still
>> working
>> > in that mode (such as the etoys derivative), it is important to leave
>> out.
>> >
>> > But, with the shiny new CI servers, couldn't we have an artifact built
>> with
>> > FFI loaded?
>> >
>> > Or, even neater, include FFI in the trunk, but have a switch/pragma in
>> the
>> > image that says 'Don't ever load FFI' for those that don't want it, but
>> > want everything else loaded?  So, for people that LIKE FFI, it is just
>> part
>> > of Trunk; for those that have to avoid it, it isn't loaded; for the
>> rest,
>> > they get whatever was setup when they got their image.  (I'd be in the
>> > first group most of the time)
>> >
>> > -cbc
>> >
>> >
>> > On Thu, Nov 14, 2013 at 2:30 PM, Frank Shearar <frank.shearar at gmail.com
>> >wrote:
>> >
>> > > On 14 November 2013 16:42, Eliot Miranda <eliot.miranda at gmail.com>
>> wrote:
>> > > > Hi Frank,
>> > > >
>> > > >
>> > > > On Thu, Nov 14, 2013 at 1:59 AM, Frank Shearar <
>> frank.shearar at gmail.com>
>> > > > wrote:
>> > > >>
>> > > >> That's the source of the problem: FFI-Kernel uses FFIConstants,
>> which
>> > > >> is declared/defined in FFI-Pools.
>> > > >>
>> > > >> Insert standard Frank rant of something called "Kernel" depending
>> on
>> > > >> other stuff. (Possibly insert standard Chris rant of a package
>> > > >> containing a single class? :) )
>> > > >
>> > > >
>> > > > The problem here is that the VM depends on FFIConstants also, and
>> the VM
>> > > > shouldn't depend on the rest of FFI.  So FFIConstants does need to
>> be on
>> > > its
>> > > > own, but could perhaps be called something different.
>> > >
>> > > You know, I would be quite happy if we folded FFI into the base image.
>> > > Seriously. For once I'd argue for _including_ something. I would
>> > > gladly trade Morphic for FFI.
>> > >
>> > > OK, I'll chalk this up to That's How It Is, and move on. If you
>> > > install from SqueakMap (as you should, and as I should have) you get
>> > > the right load order.
>> > >
>> > > frank
>> > >
>> > > >> frank
>> > > >>
>> > > >> On 12 November 2013 23:20, Chris Muller <ma.chris.m at gmail.com>
>> wrote:
>> > > >> > I didn't see you loading FFI-Pools in advance.  The issue you
>> > > >> > encountered
>> > > >> > may not have had anything to do with -eem.24.
>> > > >> >
>> > > >> > FYI -- FFI also has a "head" release on SqueakMap which
>> documents how
>> > > to
>> > > >> > load it.
>> > > >> >
>> > > >> >
>> > > >> > On Tue, Nov 12, 2013 at 4:50 PM, Frank Shearar <
>> > > frank.shearar at gmail.com>
>> > > >> > wrote:
>> > > >> >>
>> > > >> >> That pulls in the latest packages, right?
>> > > >> >>
>> > > >> >> So that does work. But why does the reported version fail to
>> load?
>> > > >> >> That's a problem still.
>> > > >> >>
>> > > >> >> (While this lets me get on with what I wanted to do - add
>> > > >> >> #interleaving: to Xtreams - this is still a problem. But thanks
>> for
>> > > >> >> the workaround, Chris!)
>> > > >> >>
>> > > >> >> frank
>> > > >> >>
>> > > >> >> On 12 November 2013 20:55, Chris Muller <asqueaker at gmail.com>
>> wrote:
>> > > >> >> > Installer new merge: #ffi.
>> > > >> >> >
>> > > >> >> > Worked for me..
>> > > >> >> >
>> > > >> >> > On Tue, Nov 12, 2013 at 2:45 PM, Frank Shearar
>> > > >> >> > <frank.shearar at gmail.com>
>> > > >> >> > wrote:
>> > > >> >> >> On 26 February 2013 18:36, Eliot Miranda <
>> eliot.miranda at gmail.com
>> > > >
>> > > >> >> >> wrote:
>> > > >> >> >>>
>> > > >> >> >>>
>> > > >> >> >>> On Tue, Feb 26, 2013 at 1:51 AM, Frank Shearar
>> > > >> >> >>> <frank.shearar at gmail.com>
>> > > >> >> >>> wrote:
>> > > >> >> >>>>
>> > > >> >> >>>> (Installer monticello mc: (MCHttpRepository new location:
>> > > >> >> >>>> 'http://source.squeak.org/FFI'))
>> > > >> >> >>>>     install: 'FFI-Kernel-eem.24.mcz'. "There's a -tbn.25,
>> but
>> > > >> >> >>>> that's
>> > > >> >> >>>> not important to this post"
>> > > >> >> >>>>
>> > > >> >> >>>> fails with an MNU: ExternalFunction class >>
>> > > >> >> >>>> callingConventionFor:.
>> > > >> >> >>>>
>> > > >> >> >>>> As far as I can see what's happening is this:
>> > > >> >> >>>> * during the loading of the mcz ExternalFunction is
>> defined,
>> > > >> >> >>>> * a method is parsed (#XOpenDisplay, which has a pragma
>> <cdecl:
>> > > >> >> >>>> X11Display* ''XOpenDisplay'' (char*) module:''X11''>)
>> > > >> >> >>>> * Parser >> externalFunctionDeclaration checks whether
>> > > >> >> >>>> ExternalFunction is defined.
>> > > >> >> >>>> * It is, so tries to evaluate `descriptorClass
>> > > callingConvention:
>> > > >> >> >>>> here` and boom, because ExternalFunction class >>
>> > > >> >> >>>> callingConventionFor: _has not been loaded yet_.
>> > > >> >> >>>
>> > > >> >> >>>
>> > > >> >> >>> I thought we'd modified Monticello to load new methods
>> first.
>> > >  Did
>> > > >> >> >>> this not
>> > > >> >> >>> get added to trunk?
>> > > >> >> >>
>> > > >> >> >> Apparently not. It still happens with an up-to-date Squeak
>> 4.5.
>> > > >> >> >>
>> > > >> >> >> frank
>> > > >> >> >>
>> > > >> >> >>>> I've seen this kind of issue with Helvetia code: sometimes
>> you
>> > > >> >> >>>> simply
>> > > >> >> >>>> have to load class-side methods first.
>> > > >> >> >>>>
>> > > >> >> >>>> Thoughts? Lukas worked around the issue with his Helvetia
>> code
>> > > by
>> > > >> >> >>>> directly patching Monticello, and ripping out its "try to
>> do
>> > > >> >> >>>> atomic
>> > > >> >> >>>> loading" mechanism.
>> > > >> >> >>>>
>> > > >> >> >>>> frank
>> > > >> >> >>>>
>> > > >> >> >>>
>> > > >> >> >>>
>> > > >> >> >>>
>> > > >> >> >>> --
>> > > >> >> >>> best,
>> > > >> >> >>> Eliot
>> > > >> >> >>>
>> > > >> >> >>>
>> > > >> >> >>>
>> > > >> >> >>
>> > > >> >> >
>> > > >> >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > best,
>> > > > Eliot
>> > >
>> > >
>>
>> >
>>
>>
>>
>
>
> --
> best,
> Eliot
>
>
>
>


-- 
Best regards,
Igor Stasenko.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20131120/0c6ac4c6/attachment.htm


More information about the Squeak-dev mailing list