[squeak-dev] Plugin Repository Needed [was FFI callbacks]

Frank Shearar frank.shearar at gmail.com
Fri Sep 4 08:26:38 UTC 2015


On 4 September 2015 at 07:01, Tobias Pape <Das.Linux at gmx.de> wrote:
>
> On 04.09.2015, at 05:46, Chris Muller <asqueaker at gmail.com> wrote:
>
>>> I do think that anything  is better than the current situation. A bucket of
>>> any kind in an agreed upon location seems like a good place to start.
>>
>> I totally agree!
>
> what about
>
> http://github.com/squeak-vm/plugins
>
> or
>
> http://github.com/squeak-smalltalk/vm-plugins

My preferred choice: the organisation already exists.

frank

> Best
>         -tobias
>
> PS: we should look at https://github.com/search?q=squeak+vm
>
>>
>>> I
>>> hadn't considered that plugins would need meta data or labels.
>>
>> They could be put into a .zip file and stored and cataloged somewhere
>> which would be a well-known place for Squeak.  And, each plugin would
>> be tagged with one or more hierarchical Category's to indicate the
>> platform and whatever else we wanted...
>>
>> And, if the plugin is compiled and hosted somewhere else, like GitHub,
>> then the Squeak-well-known place should direct the user to THAT
>> external place instead of hosting copies on squeak.org servers...
>> Except as a backup, of course..
>>
>> Hmm, if only we have something that could do all that...    Oh wait,
>> we do!!!!    ;-)
>>
>>> Perhaps the best thing is to leave it as an open question for  a few weeks
>>> before settling on any kind of solution. A common location for plugins is
>>> about as far as my thinking takes me at the moment. I'm sure the collective
>>> imagination of the community can come up with a better idea.
>>>
>>> Chris
>>>
>>>
>>> On 2015-09-03 10:07 PM, David T. Lewis wrote:
>>>>
>>>> On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote:
>>>>>
>>>>> This community needs a repository for plugins. A directory at
>>>>> http://ftp.squeak.org ought to do it.
>>>>>
>>>>> If I want to try Alien I need the plugin. Where am I going to get that?
>>>>> I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see it
>>>>> is not saying Newspeak and realize it compiles a plugin called IA32ABI,
>>>>> which - coincidentally - is the name of the plugin the error Alien
>>>>> exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>> moduleName
>>>>> returns IA32ABI).
>>>>>
>>>>> Does anybody know what the Genie plugin does? Does Paul DeBrucker need
>>>>> to store a BerkeleyDB plugin dll he got by emailing somebody on GitHub?
>>>>>
>>>>> If I understand this correctly, if such a directory existed, then all a
>>>>> person would need to do to try the above Alien code would be to download
>>>>> a pre-compiled plugin from - say -
>>>>> http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM directory
>>>>> and start the image. Then check with SmalltalkImage current
>>>>> listLoadedModules to see if you're ready to go.
>>>>>
>>>>> I nominate Fabio Niephaus to be the person the community can send
>>>>> pre-compiled plugins for inclusion in such a directory. Eliot's been
>>>>> banging away about Alien for some time [1]. If a person could go, get
>>>>> the plugin and give the code shot, he could have more productive
>>>>> conversations about Alien.
>>>>>
>>>>> So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in an
>>>>> idle moment, you could email it to Fabio? And when, Fabio, you've
>>>>> created the directory in http://ftp.squeak.org you could announce it
>>>>> here on Squeak-dev?
>>>>>
>>>>> FWIW,
>>>>>
>>>>> Chris
>>>>
>>>> A word of caution:
>>>>
>>>> I think the idea is good in principle, but it needs to be accompanied
>>>> by a commitment by someone to do the hard work of keeping track of
>>>> dependencies on the various flavors of OS, image word size, VM word size,
>>>> spur/V3, and so forth. If someone would like to take that on, great.
>>>> That said, we don't want to start a project that gets 80% completed, and
>>>> then expect "the community" to do the rest of the work.
>>>>
>>>> I cannot speak for Fabio, but I don't think it is fair for us to expect
>>>> him to take on this kind of workload unless he has indicated an interest
>>>> in doing it.
>>>>
>>>> Maybe a restricted repository focused just on compiled plugins known
>>>> to work with the Spur VM on a limited range of OS flavors would be helpful
>>>> at this stage. It probably does not belong on ftp.squeak.org, but it could
>>>> be a very useful resource if someone (Chris Cunnington?) wants to set it
>>>> up.
>>>>
>>>> Dave
>>>>
>>>>
>>>>> [1] http://wiki.squeak.org/squeak/uploads/6100/
>>>>>
>>>>> On 2015-09-03 7:07 PM, Eliot Miranda wrote:
>>>>>>
>>>>>> Hi Craig,
>>>>>>
>>>>>>    you need Alien-eem.24
>>>>>>
>>>>>> Here's Alien class>>exampleCqsort
>>>>>> "Call the libc qsort function (which requires a callback)."
>>>>>> "Alien exampleCqsort"
>>>>>> "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) /
>>>>>> 100.0"
>>>>>> | cb rand nElements sizeofDouble values orig sort |
>>>>>> rand := Random new.
>>>>>> values := Alien newC: (nElements := 100) * (sizeofDouble := 8).
>>>>>> 1 to: values dataSize by: sizeofDouble do:
>>>>>> [:i| values doubleAt: i put: rand next].
>>>>>> orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values
>>>>>> doubleAt: i].
>>>>>> cb := Callback
>>>>>> signature:  #(int (*)(const void *, const void *))
>>>>>> block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign].
>>>>>> (Alien lookup: 'qsort' inLibrary: Alien libcName)
>>>>>> primFFICallResult: nil
>>>>>> with: values pointer
>>>>>> with: nElements
>>>>>> with: sizeofDouble
>>>>>> with: cb thunk.
>>>>>> sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values
>>>>>> doubleAt: i].
>>>>>> values free.
>>>>>> ^orig -> sort
>>>>>>
>>>>>> The above example uses Alien to make the callout.  To use it with the
>>>>>> FFI simply pass cb pointer as the argument as is done above.
>>>>>>
>>>>>>
>>>>>> Implementation:
>>>>>>
>>>>>> The way that it works is in two parts
>>>>>> - the VM passes up a pointer to a structure from which all arguments,
>>>>>> stacked and in registers (because the VM has copied the register args
>>>>>> into the struct) can be accessed, and through which the result can be
>>>>>> returned.
>>>>>> - the image level provides marshalling methods that match the
>>>>>> signature in the callback
>>>>>>
>>>>>> So e.g. with a callback of
>>>>>> Callback
>>>>>> signature:  #(int (*)(const void *, const void *))
>>>>>> block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]
>>>>>> the marshalling methods are in Callback's signature protocol:
>>>>>>
>>>>>> Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien
>>>>>> <signature: #(int (*)(const void *, const void *)) abi: 'IA32'>
>>>>>> ^callbackContext wordResult:
>>>>>> (block
>>>>>> value: (Alien forPointer: (spAlien unsignedLongAt: 1))
>>>>>> value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
>>>>>>
>>>>>> where spAlien is an Alien pointing to a VMCallbackContext32.
>>>>>>
>>>>>> For ARM support we would add
>>>>>>
>>>>>> Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien
>>>>>> intRegArgs: regsAlien
>>>>>> <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'>
>>>>>> ^callbackContext wordResult:
>>>>>> (block
>>>>>> value: (Alien forPointer: (regsAlien unsignedLongAt: 1))
>>>>>> value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
>>>>>>
>>>>>> Basically the idea is that the selector of the method doesn't matter
>>>>>> except for the number of arguments.  What's important is the pragma
>>>>>> which defines the signature and the ABI for which this is a valid
>>>>>> marshalling method.
>>>>>>
>>>>>> When the callback is instantiated, Callback introspects to find the
>>>>>> marshalling method that matches the signature.  If one doesn't already
>>>>>> exist you can write one.  Hopefully we'll write an ABI compiler that
>>>>>> will automatically generate these marshalling methods according to the
>>>>>> platform's ABI, but for now its a manual process.  But at least it's
>>>>>> open and flexible.  When the callback is invoked the evaluator is
>>>>>> performed with the current callbackContext and pointer(s) to the
>>>>>> arguments. There is a 32-bit and a 64-bit callback context, and it can
>>>>>> have a stack pointer, integer register args and floating point
>>>>>> register args.  So it's general enough for any callback.
>>>>>>
>>>>>> To pass back the result, a value is assigned into the struct via the
>>>>>> accessor in the marshalling method and control returns to teh point
>>>>>> where teh callback comes in, and this uses a primitive to return.
>>>>>> Inside the callbackCOntext is a jmpbuf from a setjmp.  The primitive
>>>>>> longjmp's back to the entry point in the VM which extracts the result
>>>>>> and the code for the kind of result and returns:
>>>>>>
>>>>>> Callback class>>invokeCallbackContext: vmCallbackContextAddress
>>>>>> "<Integer>" "^<FFICallbackReturnValue>"
>>>>>> "The low-level entry-point for callbacks sent from the VM/IA32ABI
>>>>>> plugin.
>>>>>> Return via primReturnFromContext:through:.  thisContext's sender is the
>>>>>> call-out context."
>>>>>> | callbackAlien type |
>>>>>> callbackAlien := (Smalltalk wordSize = 4
>>>>>> ifTrue: [VMCallbackContext32]
>>>>>> ifFalse: [VMCallbackContext64])
>>>>>> atAddress: vmCallbackContextAddress.
>>>>>> [type := Callback evaluateCallbackForContext: callbackAlien]
>>>>>> ifCurtailed: [self error: 'attempt to non-local return across a
>>>>>> callback'].
>>>>>> type ifNil:
>>>>>> [type := 1. callbackAlien wordResult: -1].
>>>>>> callbackAlien primReturnAs: type fromContext: thisContext
>>>>>>
>>>>>> On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig at netjam.org
>>>>>> <mailto:craig at netjam.org>> wrote:
>>>>>>
>>>>>>
>>>>>>    Hi all--
>>>>>>
>>>>>>         I'd like to use a C shared library via FFI from Squeak or
>>>>>> Pharo.
>>>>>>    Specifically, I want to call a C function which takes as one of its
>>>>>>    parameters a pointer to an array of callback function addresses.
>>>>>> Does
>>>>>>    one of the FFI approaches floating around handle that, and provide a
>>>>>>    relatively pleasant mapping between Smalltalk block closures and C
>>>>>>    callback function addresses?
>>>>>>
>>>>>>         I was doing this so far in GemStone, but apparently its FFI
>>>>>> only
>>>>>>    supports passing callback function addresses one at a time as direct
>>>>>>    parameters of C callouts. I suppose I could write a wrapper C
>>>>>> library
>>>>>>    around the one I really want to use, that provides the callback
>>>>>> setup
>>>>>>    interface that GemStone expects, but whenever I have to write actual
>>>>>> C
>>>>>>    code I start to think "Hm, why don't I just use a Smalltalk VM for
>>>>>>    which
>>>>>>    I have the source?" :)  All the fancy distributed object-database
>>>>>>    stuff
>>>>>>    that my project also wants can wait for a bit.
>>>>>>
>>>>>>
>>>>>>         thanks!
>>>>>>
>>>>>>    -C
>>>>>>
>>>>>>    --
>>>>>>    Craig Latta
>>>>>>    netjam.org <http://netjam.org>
>>>>>>    +31 6 2757 7177 <tel:%2B31%20%20%206%202757%207177> (SMS ok)
>>>>>>    + 1 415 287 3547 <tel:%2B%201%20415%20%20287%203547> (no SMS)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> _,,,^..^,,,_
>>>>>> best, Eliot
>
>
>


More information about the Squeak-dev mailing list