[squeak-dev] Plugin Repository Needed [was FFI callbacks]

Chris Muller asqueaker at gmail.com
Fri Sep 4 03:46:08 UTC 2015


> I do think that anything  is better than the current situation. A bucket of
> any kind in an agreed upon location seems like a good place to start.

I totally agree!

> I
> hadn't considered that plugins would need meta data or labels.

They could be put into a .zip file and stored and cataloged somewhere
which would be a well-known place for Squeak.  And, each plugin would
be tagged with one or more hierarchical Category's to indicate the
platform and whatever else we wanted...

And, if the plugin is compiled and hosted somewhere else, like GitHub,
then the Squeak-well-known place should direct the user to THAT
external place instead of hosting copies on squeak.org servers...
Except as a backup, of course..

Hmm, if only we have something that could do all that...    Oh wait,
we do!!!!    ;-)

> Perhaps the best thing is to leave it as an open question for  a few weeks
> before settling on any kind of solution. A common location for plugins is
> about as far as my thinking takes me at the moment. I'm sure the collective
> imagination of the community can come up with a better idea.
>
> Chris
>
>
> On 2015-09-03 10:07 PM, David T. Lewis wrote:
>>
>> On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote:
>>>
>>> This community needs a repository for plugins. A directory at
>>> http://ftp.squeak.org ought to do it.
>>>
>>> If I want to try Alien I need the plugin. Where am I going to get that?
>>> I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see it
>>> is not saying Newspeak and realize it compiles a plugin called IA32ABI,
>>> which - coincidentally - is the name of the plugin the error Alien
>>> exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>> moduleName
>>> returns IA32ABI).
>>>
>>> Does anybody know what the Genie plugin does? Does Paul DeBrucker need
>>> to store a BerkeleyDB plugin dll he got by emailing somebody on GitHub?
>>>
>>> If I understand this correctly, if such a directory existed, then all a
>>> person would need to do to try the above Alien code would be to download
>>> a pre-compiled plugin from - say -
>>> http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM directory
>>> and start the image. Then check with SmalltalkImage current
>>> listLoadedModules to see if you're ready to go.
>>>
>>> I nominate Fabio Niephaus to be the person the community can send
>>> pre-compiled plugins for inclusion in such a directory. Eliot's been
>>> banging away about Alien for some time [1]. If a person could go, get
>>> the plugin and give the code shot, he could have more productive
>>> conversations about Alien.
>>>
>>> So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in an
>>> idle moment, you could email it to Fabio? And when, Fabio, you've
>>> created the directory in http://ftp.squeak.org you could announce it
>>> here on Squeak-dev?
>>>
>>> FWIW,
>>>
>>> Chris
>>
>> A word of caution:
>>
>> I think the idea is good in principle, but it needs to be accompanied
>> by a commitment by someone to do the hard work of keeping track of
>> dependencies on the various flavors of OS, image word size, VM word size,
>> spur/V3, and so forth. If someone would like to take that on, great.
>> That said, we don't want to start a project that gets 80% completed, and
>> then expect "the community" to do the rest of the work.
>>
>> I cannot speak for Fabio, but I don't think it is fair for us to expect
>> him to take on this kind of workload unless he has indicated an interest
>> in doing it.
>>
>> Maybe a restricted repository focused just on compiled plugins known
>> to work with the Spur VM on a limited range of OS flavors would be helpful
>> at this stage. It probably does not belong on ftp.squeak.org, but it could
>> be a very useful resource if someone (Chris Cunnington?) wants to set it
>> up.
>>
>> Dave
>>
>>
>>> [1] http://wiki.squeak.org/squeak/uploads/6100/
>>>
>>> On 2015-09-03 7:07 PM, Eliot Miranda wrote:
>>>>
>>>> Hi Craig,
>>>>
>>>>     you need Alien-eem.24
>>>>
>>>> Here's Alien class>>exampleCqsort
>>>> "Call the libc qsort function (which requires a callback)."
>>>> "Alien exampleCqsort"
>>>> "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) /
>>>> 100.0"
>>>> | cb rand nElements sizeofDouble values orig sort |
>>>> rand := Random new.
>>>> values := Alien newC: (nElements := 100) * (sizeofDouble := 8).
>>>> 1 to: values dataSize by: sizeofDouble do:
>>>> [:i| values doubleAt: i put: rand next].
>>>> orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values
>>>> doubleAt: i].
>>>> cb := Callback
>>>> signature:  #(int (*)(const void *, const void *))
>>>> block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign].
>>>> (Alien lookup: 'qsort' inLibrary: Alien libcName)
>>>> primFFICallResult: nil
>>>> with: values pointer
>>>> with: nElements
>>>> with: sizeofDouble
>>>> with: cb thunk.
>>>> sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values
>>>> doubleAt: i].
>>>> values free.
>>>> ^orig -> sort
>>>>
>>>> The above example uses Alien to make the callout.  To use it with the
>>>> FFI simply pass cb pointer as the argument as is done above.
>>>>
>>>>
>>>> Implementation:
>>>>
>>>> The way that it works is in two parts
>>>> - the VM passes up a pointer to a structure from which all arguments,
>>>> stacked and in registers (because the VM has copied the register args
>>>> into the struct) can be accessed, and through which the result can be
>>>> returned.
>>>> - the image level provides marshalling methods that match the
>>>> signature in the callback
>>>>
>>>> So e.g. with a callback of
>>>> Callback
>>>> signature:  #(int (*)(const void *, const void *))
>>>> block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]
>>>> the marshalling methods are in Callback's signature protocol:
>>>>
>>>> Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien
>>>> <signature: #(int (*)(const void *, const void *)) abi: 'IA32'>
>>>> ^callbackContext wordResult:
>>>> (block
>>>> value: (Alien forPointer: (spAlien unsignedLongAt: 1))
>>>> value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
>>>>
>>>> where spAlien is an Alien pointing to a VMCallbackContext32.
>>>>
>>>> For ARM support we would add
>>>>
>>>> Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien
>>>> intRegArgs: regsAlien
>>>> <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'>
>>>> ^callbackContext wordResult:
>>>> (block
>>>> value: (Alien forPointer: (regsAlien unsignedLongAt: 1))
>>>> value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
>>>>
>>>> Basically the idea is that the selector of the method doesn't matter
>>>> except for the number of arguments.  What's important is the pragma
>>>> which defines the signature and the ABI for which this is a valid
>>>> marshalling method.
>>>>
>>>> When the callback is instantiated, Callback introspects to find the
>>>> marshalling method that matches the signature.  If one doesn't already
>>>> exist you can write one.  Hopefully we'll write an ABI compiler that
>>>> will automatically generate these marshalling methods according to the
>>>> platform's ABI, but for now its a manual process.  But at least it's
>>>> open and flexible.  When the callback is invoked the evaluator is
>>>> performed with the current callbackContext and pointer(s) to the
>>>> arguments. There is a 32-bit and a 64-bit callback context, and it can
>>>> have a stack pointer, integer register args and floating point
>>>> register args.  So it's general enough for any callback.
>>>>
>>>> To pass back the result, a value is assigned into the struct via the
>>>> accessor in the marshalling method and control returns to teh point
>>>> where teh callback comes in, and this uses a primitive to return.
>>>> Inside the callbackCOntext is a jmpbuf from a setjmp.  The primitive
>>>> longjmp's back to the entry point in the VM which extracts the result
>>>> and the code for the kind of result and returns:
>>>>
>>>> Callback class>>invokeCallbackContext: vmCallbackContextAddress
>>>> "<Integer>" "^<FFICallbackReturnValue>"
>>>> "The low-level entry-point for callbacks sent from the VM/IA32ABI
>>>> plugin.
>>>> Return via primReturnFromContext:through:.  thisContext's sender is the
>>>> call-out context."
>>>> | callbackAlien type |
>>>> callbackAlien := (Smalltalk wordSize = 4
>>>> ifTrue: [VMCallbackContext32]
>>>> ifFalse: [VMCallbackContext64])
>>>> atAddress: vmCallbackContextAddress.
>>>> [type := Callback evaluateCallbackForContext: callbackAlien]
>>>> ifCurtailed: [self error: 'attempt to non-local return across a
>>>> callback'].
>>>> type ifNil:
>>>> [type := 1. callbackAlien wordResult: -1].
>>>> callbackAlien primReturnAs: type fromContext: thisContext
>>>>
>>>> On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig at netjam.org
>>>> <mailto:craig at netjam.org>> wrote:
>>>>
>>>>
>>>>     Hi all--
>>>>
>>>>          I'd like to use a C shared library via FFI from Squeak or
>>>> Pharo.
>>>>     Specifically, I want to call a C function which takes as one of its
>>>>     parameters a pointer to an array of callback function addresses.
>>>> Does
>>>>     one of the FFI approaches floating around handle that, and provide a
>>>>     relatively pleasant mapping between Smalltalk block closures and C
>>>>     callback function addresses?
>>>>
>>>>          I was doing this so far in GemStone, but apparently its FFI
>>>> only
>>>>     supports passing callback function addresses one at a time as direct
>>>>     parameters of C callouts. I suppose I could write a wrapper C
>>>> library
>>>>     around the one I really want to use, that provides the callback
>>>> setup
>>>>     interface that GemStone expects, but whenever I have to write actual
>>>> C
>>>>     code I start to think "Hm, why don't I just use a Smalltalk VM for
>>>>     which
>>>>     I have the source?" :)  All the fancy distributed object-database
>>>>     stuff
>>>>     that my project also wants can wait for a bit.
>>>>
>>>>
>>>>          thanks!
>>>>
>>>>     -C
>>>>
>>>>     --
>>>>     Craig Latta
>>>>     netjam.org <http://netjam.org>
>>>>     +31 6 2757 7177 <tel:%2B31%20%20%206%202757%207177> (SMS ok)
>>>>     + 1 415 287 3547 <tel:%2B%201%20415%20%20287%203547> (no SMS)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> _,,,^..^,,,_
>>>> best, Eliot
>>>>
>>>>
>>
>
>


More information about the Squeak-dev mailing list