[squeak-dev] Plugin Repository Needed [was FFI callbacks]

Tobias Pape Das.Linux at gmx.de
Fri Sep 4 06:01:10 UTC 2015


On 04.09.2015, at 05:46, Chris Muller <asqueaker at gmail.com> wrote:

>> I do think that anything  is better than the current situation. A bucket of
>> any kind in an agreed upon location seems like a good place to start.
> 
> I totally agree!

what about 

http://github.com/squeak-vm/plugins

or

http://github.com/squeak-smalltalk/vm-plugins

Best
	-tobias

PS: we should look at https://github.com/search?q=squeak+vm

> 
>> I
>> hadn't considered that plugins would need meta data or labels.
> 
> They could be put into a .zip file and stored and cataloged somewhere
> which would be a well-known place for Squeak.  And, each plugin would
> be tagged with one or more hierarchical Category's to indicate the
> platform and whatever else we wanted...
> 
> And, if the plugin is compiled and hosted somewhere else, like GitHub,
> then the Squeak-well-known place should direct the user to THAT
> external place instead of hosting copies on squeak.org servers...
> Except as a backup, of course..
> 
> Hmm, if only we have something that could do all that...    Oh wait,
> we do!!!!    ;-)
> 
>> Perhaps the best thing is to leave it as an open question for  a few weeks
>> before settling on any kind of solution. A common location for plugins is
>> about as far as my thinking takes me at the moment. I'm sure the collective
>> imagination of the community can come up with a better idea.
>> 
>> Chris
>> 
>> 
>> On 2015-09-03 10:07 PM, David T. Lewis wrote:
>>> 
>>> On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote:
>>>> 
>>>> This community needs a repository for plugins. A directory at
>>>> http://ftp.squeak.org ought to do it.
>>>> 
>>>> If I want to try Alien I need the plugin. Where am I going to get that?
>>>> I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see it
>>>> is not saying Newspeak and realize it compiles a plugin called IA32ABI,
>>>> which - coincidentally - is the name of the plugin the error Alien
>>>> exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>> moduleName
>>>> returns IA32ABI).
>>>> 
>>>> Does anybody know what the Genie plugin does? Does Paul DeBrucker need
>>>> to store a BerkeleyDB plugin dll he got by emailing somebody on GitHub?
>>>> 
>>>> If I understand this correctly, if such a directory existed, then all a
>>>> person would need to do to try the above Alien code would be to download
>>>> a pre-compiled plugin from - say -
>>>> http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM directory
>>>> and start the image. Then check with SmalltalkImage current
>>>> listLoadedModules to see if you're ready to go.
>>>> 
>>>> I nominate Fabio Niephaus to be the person the community can send
>>>> pre-compiled plugins for inclusion in such a directory. Eliot's been
>>>> banging away about Alien for some time [1]. If a person could go, get
>>>> the plugin and give the code shot, he could have more productive
>>>> conversations about Alien.
>>>> 
>>>> So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in an
>>>> idle moment, you could email it to Fabio? And when, Fabio, you've
>>>> created the directory in http://ftp.squeak.org you could announce it
>>>> here on Squeak-dev?
>>>> 
>>>> FWIW,
>>>> 
>>>> Chris
>>> 
>>> A word of caution:
>>> 
>>> I think the idea is good in principle, but it needs to be accompanied
>>> by a commitment by someone to do the hard work of keeping track of
>>> dependencies on the various flavors of OS, image word size, VM word size,
>>> spur/V3, and so forth. If someone would like to take that on, great.
>>> That said, we don't want to start a project that gets 80% completed, and
>>> then expect "the community" to do the rest of the work.
>>> 
>>> I cannot speak for Fabio, but I don't think it is fair for us to expect
>>> him to take on this kind of workload unless he has indicated an interest
>>> in doing it.
>>> 
>>> Maybe a restricted repository focused just on compiled plugins known
>>> to work with the Spur VM on a limited range of OS flavors would be helpful
>>> at this stage. It probably does not belong on ftp.squeak.org, but it could
>>> be a very useful resource if someone (Chris Cunnington?) wants to set it
>>> up.
>>> 
>>> Dave
>>> 
>>> 
>>>> [1] http://wiki.squeak.org/squeak/uploads/6100/
>>>> 
>>>> On 2015-09-03 7:07 PM, Eliot Miranda wrote:
>>>>> 
>>>>> Hi Craig,
>>>>> 
>>>>>    you need Alien-eem.24
>>>>> 
>>>>> Here's Alien class>>exampleCqsort
>>>>> "Call the libc qsort function (which requires a callback)."
>>>>> "Alien exampleCqsort"
>>>>> "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) /
>>>>> 100.0"
>>>>> | cb rand nElements sizeofDouble values orig sort |
>>>>> rand := Random new.
>>>>> values := Alien newC: (nElements := 100) * (sizeofDouble := 8).
>>>>> 1 to: values dataSize by: sizeofDouble do:
>>>>> [:i| values doubleAt: i put: rand next].
>>>>> orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values
>>>>> doubleAt: i].
>>>>> cb := Callback
>>>>> signature:  #(int (*)(const void *, const void *))
>>>>> block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign].
>>>>> (Alien lookup: 'qsort' inLibrary: Alien libcName)
>>>>> primFFICallResult: nil
>>>>> with: values pointer
>>>>> with: nElements
>>>>> with: sizeofDouble
>>>>> with: cb thunk.
>>>>> sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values
>>>>> doubleAt: i].
>>>>> values free.
>>>>> ^orig -> sort
>>>>> 
>>>>> The above example uses Alien to make the callout.  To use it with the
>>>>> FFI simply pass cb pointer as the argument as is done above.
>>>>> 
>>>>> 
>>>>> Implementation:
>>>>> 
>>>>> The way that it works is in two parts
>>>>> - the VM passes up a pointer to a structure from which all arguments,
>>>>> stacked and in registers (because the VM has copied the register args
>>>>> into the struct) can be accessed, and through which the result can be
>>>>> returned.
>>>>> - the image level provides marshalling methods that match the
>>>>> signature in the callback
>>>>> 
>>>>> So e.g. with a callback of
>>>>> Callback
>>>>> signature:  #(int (*)(const void *, const void *))
>>>>> block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]
>>>>> the marshalling methods are in Callback's signature protocol:
>>>>> 
>>>>> Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien
>>>>> <signature: #(int (*)(const void *, const void *)) abi: 'IA32'>
>>>>> ^callbackContext wordResult:
>>>>> (block
>>>>> value: (Alien forPointer: (spAlien unsignedLongAt: 1))
>>>>> value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
>>>>> 
>>>>> where spAlien is an Alien pointing to a VMCallbackContext32.
>>>>> 
>>>>> For ARM support we would add
>>>>> 
>>>>> Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien
>>>>> intRegArgs: regsAlien
>>>>> <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'>
>>>>> ^callbackContext wordResult:
>>>>> (block
>>>>> value: (Alien forPointer: (regsAlien unsignedLongAt: 1))
>>>>> value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
>>>>> 
>>>>> Basically the idea is that the selector of the method doesn't matter
>>>>> except for the number of arguments.  What's important is the pragma
>>>>> which defines the signature and the ABI for which this is a valid
>>>>> marshalling method.
>>>>> 
>>>>> When the callback is instantiated, Callback introspects to find the
>>>>> marshalling method that matches the signature.  If one doesn't already
>>>>> exist you can write one.  Hopefully we'll write an ABI compiler that
>>>>> will automatically generate these marshalling methods according to the
>>>>> platform's ABI, but for now its a manual process.  But at least it's
>>>>> open and flexible.  When the callback is invoked the evaluator is
>>>>> performed with the current callbackContext and pointer(s) to the
>>>>> arguments. There is a 32-bit and a 64-bit callback context, and it can
>>>>> have a stack pointer, integer register args and floating point
>>>>> register args.  So it's general enough for any callback.
>>>>> 
>>>>> To pass back the result, a value is assigned into the struct via the
>>>>> accessor in the marshalling method and control returns to teh point
>>>>> where teh callback comes in, and this uses a primitive to return.
>>>>> Inside the callbackCOntext is a jmpbuf from a setjmp.  The primitive
>>>>> longjmp's back to the entry point in the VM which extracts the result
>>>>> and the code for the kind of result and returns:
>>>>> 
>>>>> Callback class>>invokeCallbackContext: vmCallbackContextAddress
>>>>> "<Integer>" "^<FFICallbackReturnValue>"
>>>>> "The low-level entry-point for callbacks sent from the VM/IA32ABI
>>>>> plugin.
>>>>> Return via primReturnFromContext:through:.  thisContext's sender is the
>>>>> call-out context."
>>>>> | callbackAlien type |
>>>>> callbackAlien := (Smalltalk wordSize = 4
>>>>> ifTrue: [VMCallbackContext32]
>>>>> ifFalse: [VMCallbackContext64])
>>>>> atAddress: vmCallbackContextAddress.
>>>>> [type := Callback evaluateCallbackForContext: callbackAlien]
>>>>> ifCurtailed: [self error: 'attempt to non-local return across a
>>>>> callback'].
>>>>> type ifNil:
>>>>> [type := 1. callbackAlien wordResult: -1].
>>>>> callbackAlien primReturnAs: type fromContext: thisContext
>>>>> 
>>>>> On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig at netjam.org
>>>>> <mailto:craig at netjam.org>> wrote:
>>>>> 
>>>>> 
>>>>>    Hi all--
>>>>> 
>>>>>         I'd like to use a C shared library via FFI from Squeak or
>>>>> Pharo.
>>>>>    Specifically, I want to call a C function which takes as one of its
>>>>>    parameters a pointer to an array of callback function addresses.
>>>>> Does
>>>>>    one of the FFI approaches floating around handle that, and provide a
>>>>>    relatively pleasant mapping between Smalltalk block closures and C
>>>>>    callback function addresses?
>>>>> 
>>>>>         I was doing this so far in GemStone, but apparently its FFI
>>>>> only
>>>>>    supports passing callback function addresses one at a time as direct
>>>>>    parameters of C callouts. I suppose I could write a wrapper C
>>>>> library
>>>>>    around the one I really want to use, that provides the callback
>>>>> setup
>>>>>    interface that GemStone expects, but whenever I have to write actual
>>>>> C
>>>>>    code I start to think "Hm, why don't I just use a Smalltalk VM for
>>>>>    which
>>>>>    I have the source?" :)  All the fancy distributed object-database
>>>>>    stuff
>>>>>    that my project also wants can wait for a bit.
>>>>> 
>>>>> 
>>>>>         thanks!
>>>>> 
>>>>>    -C
>>>>> 
>>>>>    --
>>>>>    Craig Latta
>>>>>    netjam.org <http://netjam.org>
>>>>>    +31 6 2757 7177 <tel:%2B31%20%20%206%202757%207177> (SMS ok)
>>>>>    + 1 415 287 3547 <tel:%2B%201%20415%20%20287%203547> (no SMS)
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> _,,,^..^,,,_
>>>>> best, Eliot




More information about the Squeak-dev mailing list