Hi all--
I'd like to use a C shared library via FFI from Squeak or Pharo. Specifically, I want to call a C function which takes as one of its parameters a pointer to an array of callback function addresses. Does one of the FFI approaches floating around handle that, and provide a relatively pleasant mapping between Smalltalk block closures and C callback function addresses?
I was doing this so far in GemStone, but apparently its FFI only supports passing callback function addresses one at a time as direct parameters of C callouts. I suppose I could write a wrapper C library around the one I really want to use, that provides the callback setup interface that GemStone expects, but whenever I have to write actual C code I start to think "Hm, why don't I just use a Smalltalk VM for which I have the source?" :) All the fancy distributed object-database stuff that my project also wants can wait for a bit.
thanks!
-C
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Hi Craig,
you need Alien-eem.24
Here's Alien class>>exampleCqsort "Call the libc qsort function (which requires a callback)." "Alien exampleCqsort" "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) / 100.0" | cb rand nElements sizeofDouble values orig sort | rand := Random new. values := Alien newC: (nElements := 100) * (sizeofDouble := 8). 1 to: values dataSize by: sizeofDouble do: [:i| values doubleAt: i put: rand next]. orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. cb := Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]. (Alien lookup: 'qsort' inLibrary: Alien libcName) primFFICallResult: nil with: values pointer with: nElements with: sizeofDouble with: cb thunk. sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. values free. ^orig -> sort
The above example uses Alien to make the callout. To use it with the FFI simply pass cb pointer as the argument as is done above.
Implementation:
The way that it works is in two parts - the VM passes up a pointer to a structure from which all arguments, stacked and in registers (because the VM has copied the register args into the struct) can be accessed, and through which the result can be returned. - the image level provides marshalling methods that match the signature in the callback
So e.g. with a callback of Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign] the marshalling methods are in Callback's signature protocol:
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien <signature: #(int (*)(const void *, const void *)) abi: 'IA32'> ^callbackContext wordResult: (block value: (Alien forPointer: (spAlien unsignedLongAt: 1)) value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
where spAlien is an Alien pointing to a VMCallbackContext32.
For ARM support we would add
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien intRegArgs: regsAlien <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
Basically the idea is that the selector of the method doesn't matter except for the number of arguments. What's important is the pragma which defines the signature and the ABI for which this is a valid marshalling method.
When the callback is instantiated, Callback introspects to find the marshalling method that matches the signature. If one doesn't already exist you can write one. Hopefully we'll write an ABI compiler that will automatically generate these marshalling methods according to the platform's ABI, but for now its a manual process. But at least it's open and flexible. When the callback is invoked the evaluator is performed with the current callbackContext and pointer(s) to the arguments. There is a 32-bit and a 64-bit callback context, and it can have a stack pointer, integer register args and floating point register args. So it's general enough for any callback.
To pass back the result, a value is assigned into the struct via the accessor in the marshalling method and control returns to teh point where teh callback comes in, and this uses a primitive to return. Inside the callbackCOntext is a jmpbuf from a setjmp. The primitive longjmp's back to the entry point in the VM which extracts the result and the code for the kind of result and returns:
Callback class>>invokeCallbackContext: vmCallbackContextAddress "<Integer>" "^<FFICallbackReturnValue>" "The low-level entry-point for callbacks sent from the VM/IA32ABI plugin. Return via primReturnFromContext:through:. thisContext's sender is the call-out context." | callbackAlien type | callbackAlien := (Smalltalk wordSize = 4 ifTrue: [VMCallbackContext32] ifFalse: [VMCallbackContext64]) atAddress: vmCallbackContextAddress. [type := Callback evaluateCallbackForContext: callbackAlien] ifCurtailed: [self error: 'attempt to non-local return across a callback']. type ifNil: [type := 1. callbackAlien wordResult: -1]. callbackAlien primReturnAs: type fromContext: thisContext
On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta craig@netjam.org wrote:
Hi all--
I'd like to use a C shared library via FFI from Squeak or Pharo.Specifically, I want to call a C function which takes as one of its parameters a pointer to an array of callback function addresses. Does one of the FFI approaches floating around handle that, and provide a relatively pleasant mapping between Smalltalk block closures and C callback function addresses?
I was doing this so far in GemStone, but apparently its FFI onlysupports passing callback function addresses one at a time as direct parameters of C callouts. I suppose I could write a wrapper C library around the one I really want to use, that provides the callback setup interface that GemStone expects, but whenever I have to write actual C code I start to think "Hm, why don't I just use a Smalltalk VM for which I have the source?" :) All the fancy distributed object-database stuff that my project also wants can wait for a bit.
thanks!-C
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok)
- 1 415 287 3547 (no SMS)
This community needs a repository for plugins. A directory at http://ftp.squeak.org ought to do it.
If I want to try Alien I need the plugin. Where am I going to get that? I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see it is not saying Newspeak and realize it compiles a plugin called IA32ABI, which - coincidentally - is the name of the plugin the error Alien exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>> moduleName returns IA32ABI).
Does anybody know what the Genie plugin does? Does Paul DeBrucker need to store a BerkeleyDB plugin dll he got by emailing somebody on GitHub?
If I understand this correctly, if such a directory existed, then all a person would need to do to try the above Alien code would be to download a pre-compiled plugin from - say - http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM directory and start the image. Then check with SmalltalkImage current listLoadedModules to see if you're ready to go.
I nominate Fabio Niephaus to be the person the community can send pre-compiled plugins for inclusion in such a directory. Eliot's been banging away about Alien for some time [1]. If a person could go, get the plugin and give the code shot, he could have more productive conversations about Alien.
So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in an idle moment, you could email it to Fabio? And when, Fabio, you've created the directory in http://ftp.squeak.org you could announce it here on Squeak-dev?
FWIW,
Chris
[1] http://wiki.squeak.org/squeak/uploads/6100/
On 2015-09-03 7:07 PM, Eliot Miranda wrote:
Hi Craig,
you need Alien-eem.24Here's Alien class>>exampleCqsort "Call the libc qsort function (which requires a callback)." "Alien exampleCqsort" "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) / 100.0" | cb rand nElements sizeofDouble values orig sort | rand := Random new. values := Alien newC: (nElements := 100) * (sizeofDouble := 8). 1 to: values dataSize by: sizeofDouble do: [:i| values doubleAt: i put: rand next]. orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. cb := Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]. (Alien lookup: 'qsort' inLibrary: Alien libcName) primFFICallResult: nil with: values pointer with: nElements with: sizeofDouble with: cb thunk. sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. values free. ^orig -> sort
The above example uses Alien to make the callout. To use it with the FFI simply pass cb pointer as the argument as is done above.
Implementation:
The way that it works is in two parts
- the VM passes up a pointer to a structure from which all arguments,
stacked and in registers (because the VM has copied the register args into the struct) can be accessed, and through which the result can be returned.
- the image level provides marshalling methods that match the
signature in the callback
So e.g. with a callback of Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign] the marshalling methods are in Callback's signature protocol:
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien <signature: #(int (*)(const void *, const void *)) abi: 'IA32'> ^callbackContext wordResult: (block value: (Alien forPointer: (spAlien unsignedLongAt: 1)) value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
where spAlien is an Alien pointing to a VMCallbackContext32.
For ARM support we would add
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien intRegArgs: regsAlien <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
Basically the idea is that the selector of the method doesn't matter except for the number of arguments. What's important is the pragma which defines the signature and the ABI for which this is a valid marshalling method.
When the callback is instantiated, Callback introspects to find the marshalling method that matches the signature. If one doesn't already exist you can write one. Hopefully we'll write an ABI compiler that will automatically generate these marshalling methods according to the platform's ABI, but for now its a manual process. But at least it's open and flexible. When the callback is invoked the evaluator is performed with the current callbackContext and pointer(s) to the arguments. There is a 32-bit and a 64-bit callback context, and it can have a stack pointer, integer register args and floating point register args. So it's general enough for any callback.
To pass back the result, a value is assigned into the struct via the accessor in the marshalling method and control returns to teh point where teh callback comes in, and this uses a primitive to return. Inside the callbackCOntext is a jmpbuf from a setjmp. The primitive longjmp's back to the entry point in the VM which extracts the result and the code for the kind of result and returns:
Callback class>>invokeCallbackContext: vmCallbackContextAddress "<Integer>" "^<FFICallbackReturnValue>" "The low-level entry-point for callbacks sent from the VM/IA32ABI plugin. Return via primReturnFromContext:through:. thisContext's sender is the call-out context." | callbackAlien type | callbackAlien := (Smalltalk wordSize = 4 ifTrue: [VMCallbackContext32] ifFalse: [VMCallbackContext64]) atAddress: vmCallbackContextAddress. [type := Callback evaluateCallbackForContext: callbackAlien] ifCurtailed: [self error: 'attempt to non-local return across a callback']. type ifNil: [type := 1. callbackAlien wordResult: -1]. callbackAlien primReturnAs: type fromContext: thisContext
On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig@netjam.org mailto:craig@netjam.org> wrote:
Hi all-- I'd like to use a C shared library via FFI from Squeak or Pharo. Specifically, I want to call a C function which takes as one of its parameters a pointer to an array of callback function addresses. Does one of the FFI approaches floating around handle that, and provide a relatively pleasant mapping between Smalltalk block closures and C callback function addresses? I was doing this so far in GemStone, but apparently its FFI only supports passing callback function addresses one at a time as direct parameters of C callouts. I suppose I could write a wrapper C library around the one I really want to use, that provides the callback setup interface that GemStone expects, but whenever I have to write actual C code I start to think "Hm, why don't I just use a Smalltalk VM for which I have the source?" :) All the fancy distributed object-database stuff that my project also wants can wait for a bit. thanks! -C -- Craig Latta netjam.org <http://netjam.org> +31 6 2757 7177 <tel:%2B31%20%20%206%202757%207177> (SMS ok) + 1 415 287 3547 <tel:%2B%201%20415%20%20287%203547> (no SMS)-- _,,,^..^,,,_ best, Eliot
On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote:
This community needs a repository for plugins. A directory at http://ftp.squeak.org ought to do it.
If I want to try Alien I need the plugin. Where am I going to get that? I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see it is not saying Newspeak and realize it compiles a plugin called IA32ABI, which - coincidentally - is the name of the plugin the error Alien exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>> moduleName returns IA32ABI).
Does anybody know what the Genie plugin does? Does Paul DeBrucker need to store a BerkeleyDB plugin dll he got by emailing somebody on GitHub?
If I understand this correctly, if such a directory existed, then all a person would need to do to try the above Alien code would be to download a pre-compiled plugin from - say - http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM directory and start the image. Then check with SmalltalkImage current listLoadedModules to see if you're ready to go.
I nominate Fabio Niephaus to be the person the community can send pre-compiled plugins for inclusion in such a directory. Eliot's been banging away about Alien for some time [1]. If a person could go, get the plugin and give the code shot, he could have more productive conversations about Alien.
So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in an idle moment, you could email it to Fabio? And when, Fabio, you've created the directory in http://ftp.squeak.org you could announce it here on Squeak-dev?
FWIW,
Chris
A word of caution:
I think the idea is good in principle, but it needs to be accompanied by a commitment by someone to do the hard work of keeping track of dependencies on the various flavors of OS, image word size, VM word size, spur/V3, and so forth. If someone would like to take that on, great. That said, we don't want to start a project that gets 80% completed, and then expect "the community" to do the rest of the work.
I cannot speak for Fabio, but I don't think it is fair for us to expect him to take on this kind of workload unless he has indicated an interest in doing it.
Maybe a restricted repository focused just on compiled plugins known to work with the Spur VM on a limited range of OS flavors would be helpful at this stage. It probably does not belong on ftp.squeak.org, but it could be a very useful resource if someone (Chris Cunnington?) wants to set it up.
Dave
[1] http://wiki.squeak.org/squeak/uploads/6100/
On 2015-09-03 7:07 PM, Eliot Miranda wrote:
Hi Craig,
you need Alien-eem.24
Here's Alien class>>exampleCqsort "Call the libc qsort function (which requires a callback)." "Alien exampleCqsort" "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) / 100.0" | cb rand nElements sizeofDouble values orig sort | rand := Random new. values := Alien newC: (nElements := 100) * (sizeofDouble := 8). 1 to: values dataSize by: sizeofDouble do: [:i| values doubleAt: i put: rand next]. orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. cb := Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]. (Alien lookup: 'qsort' inLibrary: Alien libcName) primFFICallResult: nil with: values pointer with: nElements with: sizeofDouble with: cb thunk. sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. values free. ^orig -> sort
The above example uses Alien to make the callout. To use it with the FFI simply pass cb pointer as the argument as is done above.
Implementation:
The way that it works is in two parts
- the VM passes up a pointer to a structure from which all arguments,
stacked and in registers (because the VM has copied the register args into the struct) can be accessed, and through which the result can be returned.
- the image level provides marshalling methods that match the
signature in the callback
So e.g. with a callback of Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign] the marshalling methods are in Callback's signature protocol:
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien <signature: #(int (*)(const void *, const void *)) abi: 'IA32'> ^callbackContext wordResult: (block value: (Alien forPointer: (spAlien unsignedLongAt: 1)) value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
where spAlien is an Alien pointing to a VMCallbackContext32.
For ARM support we would add
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien intRegArgs: regsAlien <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
Basically the idea is that the selector of the method doesn't matter except for the number of arguments. What's important is the pragma which defines the signature and the ABI for which this is a valid marshalling method.
When the callback is instantiated, Callback introspects to find the marshalling method that matches the signature. If one doesn't already exist you can write one. Hopefully we'll write an ABI compiler that will automatically generate these marshalling methods according to the platform's ABI, but for now its a manual process. But at least it's open and flexible. When the callback is invoked the evaluator is performed with the current callbackContext and pointer(s) to the arguments. There is a 32-bit and a 64-bit callback context, and it can have a stack pointer, integer register args and floating point register args. So it's general enough for any callback.
To pass back the result, a value is assigned into the struct via the accessor in the marshalling method and control returns to teh point where teh callback comes in, and this uses a primitive to return. Inside the callbackCOntext is a jmpbuf from a setjmp. The primitive longjmp's back to the entry point in the VM which extracts the result and the code for the kind of result and returns:
Callback class>>invokeCallbackContext: vmCallbackContextAddress "<Integer>" "^<FFICallbackReturnValue>" "The low-level entry-point for callbacks sent from the VM/IA32ABI plugin. Return via primReturnFromContext:through:. thisContext's sender is the call-out context." | callbackAlien type | callbackAlien := (Smalltalk wordSize = 4 ifTrue: [VMCallbackContext32] ifFalse: [VMCallbackContext64]) atAddress: vmCallbackContextAddress. [type := Callback evaluateCallbackForContext: callbackAlien] ifCurtailed: [self error: 'attempt to non-local return across a callback']. type ifNil: [type := 1. callbackAlien wordResult: -1]. callbackAlien primReturnAs: type fromContext: thisContext
On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig@netjam.org mailto:craig@netjam.org> wrote:
Hi all--
I'd like to use a C shared library via FFI from Squeak or Pharo.Specifically, I want to call a C function which takes as one of its parameters a pointer to an array of callback function addresses. Does one of the FFI approaches floating around handle that, and provide a relatively pleasant mapping between Smalltalk block closures and C callback function addresses?
I was doing this so far in GemStone, but apparently its FFI onlysupports passing callback function addresses one at a time as direct parameters of C callouts. I suppose I could write a wrapper C library around the one I really want to use, that provides the callback setup interface that GemStone expects, but whenever I have to write actual C code I start to think "Hm, why don't I just use a Smalltalk VM for which I have the source?" :) All the fancy distributed object-database stuff that my project also wants can wait for a bit.
thanks!-C
-- Craig Latta netjam.org http://netjam.org +31 6 2757 7177 tel:%2B31%20%20%206%202757%207177 (SMS ok)
- 1 415 287 3547 tel:%2B%201%20415%20%20287%203547 (no SMS)
-- _,,,^..^,,,_ best, Eliot
I'm certainly interested in participating in something that addresses this situation.
I do think that anything is better than the current situation. A bucket of any kind in an agreed upon location seems like a good place to start. I hadn't considered that plugins would need meta data or labels.
Perhaps the best thing is to leave it as an open question for a few weeks before settling on any kind of solution. A common location for plugins is about as far as my thinking takes me at the moment. I'm sure the collective imagination of the community can come up with a better idea.
Chris
On 2015-09-03 10:07 PM, David T. Lewis wrote:
On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote:
This community needs a repository for plugins. A directory at http://ftp.squeak.org ought to do it.
If I want to try Alien I need the plugin. Where am I going to get that? I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see it is not saying Newspeak and realize it compiles a plugin called IA32ABI, which - coincidentally - is the name of the plugin the error Alien exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>> moduleName returns IA32ABI).
Does anybody know what the Genie plugin does? Does Paul DeBrucker need to store a BerkeleyDB plugin dll he got by emailing somebody on GitHub?
If I understand this correctly, if such a directory existed, then all a person would need to do to try the above Alien code would be to download a pre-compiled plugin from - say - http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM directory and start the image. Then check with SmalltalkImage current listLoadedModules to see if you're ready to go.
I nominate Fabio Niephaus to be the person the community can send pre-compiled plugins for inclusion in such a directory. Eliot's been banging away about Alien for some time [1]. If a person could go, get the plugin and give the code shot, he could have more productive conversations about Alien.
So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in an idle moment, you could email it to Fabio? And when, Fabio, you've created the directory in http://ftp.squeak.org you could announce it here on Squeak-dev?
FWIW,
Chris
A word of caution:
I think the idea is good in principle, but it needs to be accompanied by a commitment by someone to do the hard work of keeping track of dependencies on the various flavors of OS, image word size, VM word size, spur/V3, and so forth. If someone would like to take that on, great. That said, we don't want to start a project that gets 80% completed, and then expect "the community" to do the rest of the work.
I cannot speak for Fabio, but I don't think it is fair for us to expect him to take on this kind of workload unless he has indicated an interest in doing it.
Maybe a restricted repository focused just on compiled plugins known to work with the Spur VM on a limited range of OS flavors would be helpful at this stage. It probably does not belong on ftp.squeak.org, but it could be a very useful resource if someone (Chris Cunnington?) wants to set it up.
Dave
[1] http://wiki.squeak.org/squeak/uploads/6100/
On 2015-09-03 7:07 PM, Eliot Miranda wrote:
Hi Craig,
you need Alien-eem.24Here's Alien class>>exampleCqsort "Call the libc qsort function (which requires a callback)." "Alien exampleCqsort" "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) / 100.0" | cb rand nElements sizeofDouble values orig sort | rand := Random new. values := Alien newC: (nElements := 100) * (sizeofDouble := 8). 1 to: values dataSize by: sizeofDouble do: [:i| values doubleAt: i put: rand next]. orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. cb := Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]. (Alien lookup: 'qsort' inLibrary: Alien libcName) primFFICallResult: nil with: values pointer with: nElements with: sizeofDouble with: cb thunk. sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. values free. ^orig -> sort
The above example uses Alien to make the callout. To use it with the FFI simply pass cb pointer as the argument as is done above.
Implementation:
The way that it works is in two parts
- the VM passes up a pointer to a structure from which all arguments,
stacked and in registers (because the VM has copied the register args into the struct) can be accessed, and through which the result can be returned.
- the image level provides marshalling methods that match the
signature in the callback
So e.g. with a callback of Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign] the marshalling methods are in Callback's signature protocol:
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien <signature: #(int (*)(const void *, const void *)) abi: 'IA32'> ^callbackContext wordResult: (block value: (Alien forPointer: (spAlien unsignedLongAt: 1)) value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
where spAlien is an Alien pointing to a VMCallbackContext32.
For ARM support we would add
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien intRegArgs: regsAlien <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
Basically the idea is that the selector of the method doesn't matter except for the number of arguments. What's important is the pragma which defines the signature and the ABI for which this is a valid marshalling method.
When the callback is instantiated, Callback introspects to find the marshalling method that matches the signature. If one doesn't already exist you can write one. Hopefully we'll write an ABI compiler that will automatically generate these marshalling methods according to the platform's ABI, but for now its a manual process. But at least it's open and flexible. When the callback is invoked the evaluator is performed with the current callbackContext and pointer(s) to the arguments. There is a 32-bit and a 64-bit callback context, and it can have a stack pointer, integer register args and floating point register args. So it's general enough for any callback.
To pass back the result, a value is assigned into the struct via the accessor in the marshalling method and control returns to teh point where teh callback comes in, and this uses a primitive to return. Inside the callbackCOntext is a jmpbuf from a setjmp. The primitive longjmp's back to the entry point in the VM which extracts the result and the code for the kind of result and returns:
Callback class>>invokeCallbackContext: vmCallbackContextAddress "<Integer>" "^<FFICallbackReturnValue>" "The low-level entry-point for callbacks sent from the VM/IA32ABI plugin. Return via primReturnFromContext:through:. thisContext's sender is the call-out context." | callbackAlien type | callbackAlien := (Smalltalk wordSize = 4 ifTrue: [VMCallbackContext32] ifFalse: [VMCallbackContext64]) atAddress: vmCallbackContextAddress. [type := Callback evaluateCallbackForContext: callbackAlien] ifCurtailed: [self error: 'attempt to non-local return across a callback']. type ifNil: [type := 1. callbackAlien wordResult: -1]. callbackAlien primReturnAs: type fromContext: thisContext
On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig@netjam.org mailto:craig@netjam.org> wrote:
Hi all-- I'd like to use a C shared library via FFI from Squeak or Pharo. Specifically, I want to call a C function which takes as one of its parameters a pointer to an array of callback function addresses. Does one of the FFI approaches floating around handle that, and provide a relatively pleasant mapping between Smalltalk block closures and C callback function addresses? I was doing this so far in GemStone, but apparently its FFI only supports passing callback function addresses one at a time as direct parameters of C callouts. I suppose I could write a wrapper C library around the one I really want to use, that provides the callback setup interface that GemStone expects, but whenever I have to write actual C code I start to think "Hm, why don't I just use a Smalltalk VM for which I have the source?" :) All the fancy distributed object-database stuff that my project also wants can wait for a bit. thanks! -C -- Craig Latta netjam.org <http://netjam.org> +31 6 2757 7177 <tel:%2B31%20%20%206%202757%207177> (SMS ok) + 1 415 287 3547 <tel:%2B%201%20415%20%20287%203547> (no SMS)-- _,,,^..^,,,_ best, Eliot
I do think that anything is better than the current situation. A bucket of any kind in an agreed upon location seems like a good place to start.
I totally agree!
I hadn't considered that plugins would need meta data or labels.
They could be put into a .zip file and stored and cataloged somewhere which would be a well-known place for Squeak. And, each plugin would be tagged with one or more hierarchical Category's to indicate the platform and whatever else we wanted...
And, if the plugin is compiled and hosted somewhere else, like GitHub, then the Squeak-well-known place should direct the user to THAT external place instead of hosting copies on squeak.org servers... Except as a backup, of course..
Hmm, if only we have something that could do all that... Oh wait, we do!!!! ;-)
Perhaps the best thing is to leave it as an open question for a few weeks before settling on any kind of solution. A common location for plugins is about as far as my thinking takes me at the moment. I'm sure the collective imagination of the community can come up with a better idea.
Chris
On 2015-09-03 10:07 PM, David T. Lewis wrote:
On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote:
This community needs a repository for plugins. A directory at http://ftp.squeak.org ought to do it.
If I want to try Alien I need the plugin. Where am I going to get that? I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see it is not saying Newspeak and realize it compiles a plugin called IA32ABI, which - coincidentally - is the name of the plugin the error Alien exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>> moduleName returns IA32ABI).
Does anybody know what the Genie plugin does? Does Paul DeBrucker need to store a BerkeleyDB plugin dll he got by emailing somebody on GitHub?
If I understand this correctly, if such a directory existed, then all a person would need to do to try the above Alien code would be to download a pre-compiled plugin from - say - http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM directory and start the image. Then check with SmalltalkImage current listLoadedModules to see if you're ready to go.
I nominate Fabio Niephaus to be the person the community can send pre-compiled plugins for inclusion in such a directory. Eliot's been banging away about Alien for some time [1]. If a person could go, get the plugin and give the code shot, he could have more productive conversations about Alien.
So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in an idle moment, you could email it to Fabio? And when, Fabio, you've created the directory in http://ftp.squeak.org you could announce it here on Squeak-dev?
FWIW,
Chris
A word of caution:
I think the idea is good in principle, but it needs to be accompanied by a commitment by someone to do the hard work of keeping track of dependencies on the various flavors of OS, image word size, VM word size, spur/V3, and so forth. If someone would like to take that on, great. That said, we don't want to start a project that gets 80% completed, and then expect "the community" to do the rest of the work.
I cannot speak for Fabio, but I don't think it is fair for us to expect him to take on this kind of workload unless he has indicated an interest in doing it.
Maybe a restricted repository focused just on compiled plugins known to work with the Spur VM on a limited range of OS flavors would be helpful at this stage. It probably does not belong on ftp.squeak.org, but it could be a very useful resource if someone (Chris Cunnington?) wants to set it up.
Dave
[1] http://wiki.squeak.org/squeak/uploads/6100/
On 2015-09-03 7:07 PM, Eliot Miranda wrote:
Hi Craig,
you need Alien-eem.24Here's Alien class>>exampleCqsort "Call the libc qsort function (which requires a callback)." "Alien exampleCqsort" "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) / 100.0" | cb rand nElements sizeofDouble values orig sort | rand := Random new. values := Alien newC: (nElements := 100) * (sizeofDouble := 8). 1 to: values dataSize by: sizeofDouble do: [:i| values doubleAt: i put: rand next]. orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. cb := Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]. (Alien lookup: 'qsort' inLibrary: Alien libcName) primFFICallResult: nil with: values pointer with: nElements with: sizeofDouble with: cb thunk. sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. values free. ^orig -> sort
The above example uses Alien to make the callout. To use it with the FFI simply pass cb pointer as the argument as is done above.
Implementation:
The way that it works is in two parts
- the VM passes up a pointer to a structure from which all arguments,
stacked and in registers (because the VM has copied the register args into the struct) can be accessed, and through which the result can be returned.
- the image level provides marshalling methods that match the
signature in the callback
So e.g. with a callback of Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign] the marshalling methods are in Callback's signature protocol:
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien <signature: #(int (*)(const void *, const void *)) abi: 'IA32'> ^callbackContext wordResult: (block value: (Alien forPointer: (spAlien unsignedLongAt: 1)) value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
where spAlien is an Alien pointing to a VMCallbackContext32.
For ARM support we would add
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien intRegArgs: regsAlien <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
Basically the idea is that the selector of the method doesn't matter except for the number of arguments. What's important is the pragma which defines the signature and the ABI for which this is a valid marshalling method.
When the callback is instantiated, Callback introspects to find the marshalling method that matches the signature. If one doesn't already exist you can write one. Hopefully we'll write an ABI compiler that will automatically generate these marshalling methods according to the platform's ABI, but for now its a manual process. But at least it's open and flexible. When the callback is invoked the evaluator is performed with the current callbackContext and pointer(s) to the arguments. There is a 32-bit and a 64-bit callback context, and it can have a stack pointer, integer register args and floating point register args. So it's general enough for any callback.
To pass back the result, a value is assigned into the struct via the accessor in the marshalling method and control returns to teh point where teh callback comes in, and this uses a primitive to return. Inside the callbackCOntext is a jmpbuf from a setjmp. The primitive longjmp's back to the entry point in the VM which extracts the result and the code for the kind of result and returns:
Callback class>>invokeCallbackContext: vmCallbackContextAddress "<Integer>" "^<FFICallbackReturnValue>" "The low-level entry-point for callbacks sent from the VM/IA32ABI plugin. Return via primReturnFromContext:through:. thisContext's sender is the call-out context." | callbackAlien type | callbackAlien := (Smalltalk wordSize = 4 ifTrue: [VMCallbackContext32] ifFalse: [VMCallbackContext64]) atAddress: vmCallbackContextAddress. [type := Callback evaluateCallbackForContext: callbackAlien] ifCurtailed: [self error: 'attempt to non-local return across a callback']. type ifNil: [type := 1. callbackAlien wordResult: -1]. callbackAlien primReturnAs: type fromContext: thisContext
On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig@netjam.org mailto:craig@netjam.org> wrote:
Hi all-- I'd like to use a C shared library via FFI from Squeak orPharo. Specifically, I want to call a C function which takes as one of its parameters a pointer to an array of callback function addresses. Does one of the FFI approaches floating around handle that, and provide a relatively pleasant mapping between Smalltalk block closures and C callback function addresses?
I was doing this so far in GemStone, but apparently its FFIonly supports passing callback function addresses one at a time as direct parameters of C callouts. I suppose I could write a wrapper C library around the one I really want to use, that provides the callback setup interface that GemStone expects, but whenever I have to write actual C code I start to think "Hm, why don't I just use a Smalltalk VM for which I have the source?" :) All the fancy distributed object-database stuff that my project also wants can wait for a bit.
thanks! -C -- Craig Latta netjam.org <http://netjam.org> +31 6 2757 7177 <tel:%2B31%20%20%206%202757%207177> (SMS ok) + 1 415 287 3547 <tel:%2B%201%20415%20%20287%203547> (no SMS)-- _,,,^..^,,,_ best, Eliot
On 04.09.2015, at 05:46, Chris Muller asqueaker@gmail.com wrote:
I do think that anything is better than the current situation. A bucket of any kind in an agreed upon location seems like a good place to start.
I totally agree!
what about
http://github.com/squeak-vm/plugins
or
http://github.com/squeak-smalltalk/vm-plugins
Best -tobias
PS: we should look at https://github.com/search?q=squeak+vm
I hadn't considered that plugins would need meta data or labels.
They could be put into a .zip file and stored and cataloged somewhere which would be a well-known place for Squeak. And, each plugin would be tagged with one or more hierarchical Category's to indicate the platform and whatever else we wanted...
And, if the plugin is compiled and hosted somewhere else, like GitHub, then the Squeak-well-known place should direct the user to THAT external place instead of hosting copies on squeak.org servers... Except as a backup, of course..
Hmm, if only we have something that could do all that... Oh wait, we do!!!! ;-)
Perhaps the best thing is to leave it as an open question for a few weeks before settling on any kind of solution. A common location for plugins is about as far as my thinking takes me at the moment. I'm sure the collective imagination of the community can come up with a better idea.
Chris
On 2015-09-03 10:07 PM, David T. Lewis wrote:
On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote:
This community needs a repository for plugins. A directory at http://ftp.squeak.org ought to do it.
If I want to try Alien I need the plugin. Where am I going to get that? I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see it is not saying Newspeak and realize it compiles a plugin called IA32ABI, which - coincidentally - is the name of the plugin the error Alien exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>> moduleName returns IA32ABI).
Does anybody know what the Genie plugin does? Does Paul DeBrucker need to store a BerkeleyDB plugin dll he got by emailing somebody on GitHub?
If I understand this correctly, if such a directory existed, then all a person would need to do to try the above Alien code would be to download a pre-compiled plugin from - say - http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM directory and start the image. Then check with SmalltalkImage current listLoadedModules to see if you're ready to go.
I nominate Fabio Niephaus to be the person the community can send pre-compiled plugins for inclusion in such a directory. Eliot's been banging away about Alien for some time [1]. If a person could go, get the plugin and give the code shot, he could have more productive conversations about Alien.
So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in an idle moment, you could email it to Fabio? And when, Fabio, you've created the directory in http://ftp.squeak.org you could announce it here on Squeak-dev?
FWIW,
Chris
A word of caution:
I think the idea is good in principle, but it needs to be accompanied by a commitment by someone to do the hard work of keeping track of dependencies on the various flavors of OS, image word size, VM word size, spur/V3, and so forth. If someone would like to take that on, great. That said, we don't want to start a project that gets 80% completed, and then expect "the community" to do the rest of the work.
I cannot speak for Fabio, but I don't think it is fair for us to expect him to take on this kind of workload unless he has indicated an interest in doing it.
Maybe a restricted repository focused just on compiled plugins known to work with the Spur VM on a limited range of OS flavors would be helpful at this stage. It probably does not belong on ftp.squeak.org, but it could be a very useful resource if someone (Chris Cunnington?) wants to set it up.
Dave
[1] http://wiki.squeak.org/squeak/uploads/6100/
On 2015-09-03 7:07 PM, Eliot Miranda wrote:
Hi Craig,
you need Alien-eem.24
Here's Alien class>>exampleCqsort "Call the libc qsort function (which requires a callback)." "Alien exampleCqsort" "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) / 100.0" | cb rand nElements sizeofDouble values orig sort | rand := Random new. values := Alien newC: (nElements := 100) * (sizeofDouble := 8). 1 to: values dataSize by: sizeofDouble do: [:i| values doubleAt: i put: rand next]. orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. cb := Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]. (Alien lookup: 'qsort' inLibrary: Alien libcName) primFFICallResult: nil with: values pointer with: nElements with: sizeofDouble with: cb thunk. sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. values free. ^orig -> sort
The above example uses Alien to make the callout. To use it with the FFI simply pass cb pointer as the argument as is done above.
Implementation:
The way that it works is in two parts
- the VM passes up a pointer to a structure from which all arguments,
stacked and in registers (because the VM has copied the register args into the struct) can be accessed, and through which the result can be returned.
- the image level provides marshalling methods that match the
signature in the callback
So e.g. with a callback of Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign] the marshalling methods are in Callback's signature protocol:
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien <signature: #(int (*)(const void *, const void *)) abi: 'IA32'> ^callbackContext wordResult: (block value: (Alien forPointer: (spAlien unsignedLongAt: 1)) value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
where spAlien is an Alien pointing to a VMCallbackContext32.
For ARM support we would add
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien intRegArgs: regsAlien <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
Basically the idea is that the selector of the method doesn't matter except for the number of arguments. What's important is the pragma which defines the signature and the ABI for which this is a valid marshalling method.
When the callback is instantiated, Callback introspects to find the marshalling method that matches the signature. If one doesn't already exist you can write one. Hopefully we'll write an ABI compiler that will automatically generate these marshalling methods according to the platform's ABI, but for now its a manual process. But at least it's open and flexible. When the callback is invoked the evaluator is performed with the current callbackContext and pointer(s) to the arguments. There is a 32-bit and a 64-bit callback context, and it can have a stack pointer, integer register args and floating point register args. So it's general enough for any callback.
To pass back the result, a value is assigned into the struct via the accessor in the marshalling method and control returns to teh point where teh callback comes in, and this uses a primitive to return. Inside the callbackCOntext is a jmpbuf from a setjmp. The primitive longjmp's back to the entry point in the VM which extracts the result and the code for the kind of result and returns:
Callback class>>invokeCallbackContext: vmCallbackContextAddress "<Integer>" "^<FFICallbackReturnValue>" "The low-level entry-point for callbacks sent from the VM/IA32ABI plugin. Return via primReturnFromContext:through:. thisContext's sender is the call-out context." | callbackAlien type | callbackAlien := (Smalltalk wordSize = 4 ifTrue: [VMCallbackContext32] ifFalse: [VMCallbackContext64]) atAddress: vmCallbackContextAddress. [type := Callback evaluateCallbackForContext: callbackAlien] ifCurtailed: [self error: 'attempt to non-local return across a callback']. type ifNil: [type := 1. callbackAlien wordResult: -1]. callbackAlien primReturnAs: type fromContext: thisContext
On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig@netjam.org mailto:craig@netjam.org> wrote:
Hi all--
I'd like to use a C shared library via FFI from Squeak orPharo. Specifically, I want to call a C function which takes as one of its parameters a pointer to an array of callback function addresses. Does one of the FFI approaches floating around handle that, and provide a relatively pleasant mapping between Smalltalk block closures and C callback function addresses?
I was doing this so far in GemStone, but apparently its FFIonly supports passing callback function addresses one at a time as direct parameters of C callouts. I suppose I could write a wrapper C library around the one I really want to use, that provides the callback setup interface that GemStone expects, but whenever I have to write actual C code I start to think "Hm, why don't I just use a Smalltalk VM for which I have the source?" :) All the fancy distributed object-database stuff that my project also wants can wait for a bit.
thanks!-C
-- Craig Latta netjam.org http://netjam.org +31 6 2757 7177 tel:%2B31%20%20%206%202757%207177 (SMS ok)
- 1 415 287 3547 tel:%2B%201%20415%20%20287%203547 (no SMS)
-- _,,,^..^,,,_ best, Eliot
On 4 September 2015 at 07:01, Tobias Pape Das.Linux@gmx.de wrote:
On 04.09.2015, at 05:46, Chris Muller asqueaker@gmail.com wrote:
I do think that anything is better than the current situation. A bucket of any kind in an agreed upon location seems like a good place to start.
I totally agree!
what about
http://github.com/squeak-vm/plugins
or
My preferred choice: the organisation already exists.
frank
Best -tobias
PS: we should look at https://github.com/search?q=squeak+vm
I hadn't considered that plugins would need meta data or labels.
They could be put into a .zip file and stored and cataloged somewhere which would be a well-known place for Squeak. And, each plugin would be tagged with one or more hierarchical Category's to indicate the platform and whatever else we wanted...
And, if the plugin is compiled and hosted somewhere else, like GitHub, then the Squeak-well-known place should direct the user to THAT external place instead of hosting copies on squeak.org servers... Except as a backup, of course..
Hmm, if only we have something that could do all that... Oh wait, we do!!!! ;-)
Perhaps the best thing is to leave it as an open question for a few weeks before settling on any kind of solution. A common location for plugins is about as far as my thinking takes me at the moment. I'm sure the collective imagination of the community can come up with a better idea.
Chris
On 2015-09-03 10:07 PM, David T. Lewis wrote:
On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote:
This community needs a repository for plugins. A directory at http://ftp.squeak.org ought to do it.
If I want to try Alien I need the plugin. Where am I going to get that? I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see it is not saying Newspeak and realize it compiles a plugin called IA32ABI, which - coincidentally - is the name of the plugin the error Alien exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>> moduleName returns IA32ABI).
Does anybody know what the Genie plugin does? Does Paul DeBrucker need to store a BerkeleyDB plugin dll he got by emailing somebody on GitHub?
If I understand this correctly, if such a directory existed, then all a person would need to do to try the above Alien code would be to download a pre-compiled plugin from - say - http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM directory and start the image. Then check with SmalltalkImage current listLoadedModules to see if you're ready to go.
I nominate Fabio Niephaus to be the person the community can send pre-compiled plugins for inclusion in such a directory. Eliot's been banging away about Alien for some time [1]. If a person could go, get the plugin and give the code shot, he could have more productive conversations about Alien.
So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in an idle moment, you could email it to Fabio? And when, Fabio, you've created the directory in http://ftp.squeak.org you could announce it here on Squeak-dev?
FWIW,
Chris
A word of caution:
I think the idea is good in principle, but it needs to be accompanied by a commitment by someone to do the hard work of keeping track of dependencies on the various flavors of OS, image word size, VM word size, spur/V3, and so forth. If someone would like to take that on, great. That said, we don't want to start a project that gets 80% completed, and then expect "the community" to do the rest of the work.
I cannot speak for Fabio, but I don't think it is fair for us to expect him to take on this kind of workload unless he has indicated an interest in doing it.
Maybe a restricted repository focused just on compiled plugins known to work with the Spur VM on a limited range of OS flavors would be helpful at this stage. It probably does not belong on ftp.squeak.org, but it could be a very useful resource if someone (Chris Cunnington?) wants to set it up.
Dave
[1] http://wiki.squeak.org/squeak/uploads/6100/
On 2015-09-03 7:07 PM, Eliot Miranda wrote:
Hi Craig,
you need Alien-eem.24
Here's Alien class>>exampleCqsort "Call the libc qsort function (which requires a callback)." "Alien exampleCqsort" "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]]) / 100.0" | cb rand nElements sizeofDouble values orig sort | rand := Random new. values := Alien newC: (nElements := 100) * (sizeofDouble := 8). 1 to: values dataSize by: sizeofDouble do: [:i| values doubleAt: i put: rand next]. orig := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. cb := Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign]. (Alien lookup: 'qsort' inLibrary: Alien libcName) primFFICallResult: nil with: values pointer with: nElements with: sizeofDouble with: cb thunk. sort := (1 to: values dataSize by: sizeofDouble) collect: [:i| values doubleAt: i]. values free. ^orig -> sort
The above example uses Alien to make the callout. To use it with the FFI simply pass cb pointer as the argument as is done above.
Implementation:
The way that it works is in two parts
- the VM passes up a pointer to a structure from which all arguments,
stacked and in registers (because the VM has copied the register args into the struct) can be accessed, and through which the result can be returned.
- the image level provides marshalling methods that match the
signature in the callback
So e.g. with a callback of Callback signature: #(int (*)(const void *, const void *)) block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1)) sign] the marshalling methods are in Callback's signature protocol:
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien <signature: #(int (*)(const void *, const void *)) abi: 'IA32'> ^callbackContext wordResult: (block value: (Alien forPointer: (spAlien unsignedLongAt: 1)) value: (Alien forPointer: (spAlien unsignedLongAt: 5)))
where spAlien is an Alien pointing to a VMCallbackContext32.
For ARM support we would add
Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien intRegArgs: regsAlien <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'> ^callbackContext wordResult: (block value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) value: (Alien forPointer: (regsAlien unsignedLongAt: 5)))
Basically the idea is that the selector of the method doesn't matter except for the number of arguments. What's important is the pragma which defines the signature and the ABI for which this is a valid marshalling method.
When the callback is instantiated, Callback introspects to find the marshalling method that matches the signature. If one doesn't already exist you can write one. Hopefully we'll write an ABI compiler that will automatically generate these marshalling methods according to the platform's ABI, but for now its a manual process. But at least it's open and flexible. When the callback is invoked the evaluator is performed with the current callbackContext and pointer(s) to the arguments. There is a 32-bit and a 64-bit callback context, and it can have a stack pointer, integer register args and floating point register args. So it's general enough for any callback.
To pass back the result, a value is assigned into the struct via the accessor in the marshalling method and control returns to teh point where teh callback comes in, and this uses a primitive to return. Inside the callbackCOntext is a jmpbuf from a setjmp. The primitive longjmp's back to the entry point in the VM which extracts the result and the code for the kind of result and returns:
Callback class>>invokeCallbackContext: vmCallbackContextAddress "<Integer>" "^<FFICallbackReturnValue>" "The low-level entry-point for callbacks sent from the VM/IA32ABI plugin. Return via primReturnFromContext:through:. thisContext's sender is the call-out context." | callbackAlien type | callbackAlien := (Smalltalk wordSize = 4 ifTrue: [VMCallbackContext32] ifFalse: [VMCallbackContext64]) atAddress: vmCallbackContextAddress. [type := Callback evaluateCallbackForContext: callbackAlien] ifCurtailed: [self error: 'attempt to non-local return across a callback']. type ifNil: [type := 1. callbackAlien wordResult: -1]. callbackAlien primReturnAs: type fromContext: thisContext
On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig@netjam.org mailto:craig@netjam.org> wrote:
Hi all--
I'd like to use a C shared library via FFI from Squeak orPharo. Specifically, I want to call a C function which takes as one of its parameters a pointer to an array of callback function addresses. Does one of the FFI approaches floating around handle that, and provide a relatively pleasant mapping between Smalltalk block closures and C callback function addresses?
I was doing this so far in GemStone, but apparently its FFIonly supports passing callback function addresses one at a time as direct parameters of C callouts. I suppose I could write a wrapper C library around the one I really want to use, that provides the callback setup interface that GemStone expects, but whenever I have to write actual C code I start to think "Hm, why don't I just use a Smalltalk VM for which I have the source?" :) All the fancy distributed object-database stuff that my project also wants can wait for a bit.
thanks!-C
-- Craig Latta netjam.org http://netjam.org +31 6 2757 7177 tel:%2B31%20%20%206%202757%207177 (SMS ok)
- 1 415 287 3547 tel:%2B%201%20415%20%20287%203547 (no SMS)
-- _,,,^..^,,,_ best, Eliot
On Fri, Sep 4, 2015 at 9:26 AM Frank Shearar frank.shearar@gmail.com wrote:
On 4 September 2015 at 07:01, Tobias Pape Das.Linux@gmx.de wrote:
On 04.09.2015, at 05:46, Chris Muller asqueaker@gmail.com wrote:
I do think that anything is better than the current situation. A
bucket of
any kind in an agreed upon location seems like a good place to start.
I totally agree!
what about
http://github.com/squeak-vm/plugins
or
My preferred choice: the organisation already exists.
frank
Best -tobias
PS: we should look at https://github.com/search?q=squeak+vm
I hadn't considered that plugins would need meta data or labels.
They could be put into a .zip file and stored and cataloged somewhere which would be a well-known place for Squeak. And, each plugin would be tagged with one or more hierarchical Category's to indicate the platform and whatever else we wanted...
And, if the plugin is compiled and hosted somewhere else, like GitHub, then the Squeak-well-known place should direct the user to THAT external place instead of hosting copies on squeak.org servers... Except as a backup, of course..
Hmm, if only we have something that could do all that... Oh wait, we do!!!! ;-)
Perhaps the best thing is to leave it as an open question for a few
weeks
before settling on any kind of solution. A common location for plugins
is
about as far as my thinking takes me at the moment. I'm sure the
collective
imagination of the community can come up with a better idea.
Chris
On 2015-09-03 10:07 PM, David T. Lewis wrote:
On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote:
This community needs a repository for plugins. A directory at http://ftp.squeak.org ought to do it.
If I want to try Alien I need the plugin. Where am I going to get
that?
I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see
it
is not saying Newspeak and realize it compiles a plugin called
IA32ABI,
which - coincidentally - is the name of the plugin the error Alien exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>>
moduleName
returns IA32ABI).
Does anybody know what the Genie plugin does? Does Paul DeBrucker
need
to store a BerkeleyDB plugin dll he got by emailing somebody on
GitHub?
If I understand this correctly, if such a directory existed, then
all a
person would need to do to try the above Alien code would be to
download
a pre-compiled plugin from - say - http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM
directory
and start the image. Then check with SmalltalkImage current listLoadedModules to see if you're ready to go.
I nominate Fabio Niephaus to be the person the community can send pre-compiled plugins for inclusion in such a directory. Eliot's been banging away about Alien for some time [1]. If a person could go, get the plugin and give the code shot, he could have more productive conversations about Alien.
So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in
an
idle moment, you could email it to Fabio? And when, Fabio, you've created the directory in http://ftp.squeak.org you could announce it here on Squeak-dev?
FWIW,
Chris
A word of caution:
I think the idea is good in principle, but it needs to be accompanied by a commitment by someone to do the hard work of keeping track of dependencies on the various flavors of OS, image word size, VM word
size,
spur/V3, and so forth. If someone would like to take that on, great. That said, we don't want to start a project that gets 80% completed,
and
then expect "the community" to do the rest of the work.
I cannot speak for Fabio, but I don't think it is fair for us to
expect
him to take on this kind of workload unless he has indicated an
interest
in doing it.
Maybe a restricted repository focused just on compiled plugins known to work with the Spur VM on a limited range of OS flavors would be
helpful
at this stage. It probably does not belong on ftp.squeak.org, but it
could
be a very useful resource if someone (Chris Cunnington?) wants to set
it
up.
Dave
[1] http://wiki.squeak.org/squeak/uploads/6100/
On 2015-09-03 7:07 PM, Eliot Miranda wrote: > > Hi Craig, > > you need Alien-eem.24 > > Here's Alien class>>exampleCqsort > "Call the libc qsort function (which requires a callback)." > "Alien exampleCqsort" > "(Time millisecondsToRun: [100 timesRepeat: [Alien exampleCqsort]])
/
> 100.0" > | cb rand nElements sizeofDouble values orig sort | > rand := Random new. > values := Alien newC: (nElements := 100) * (sizeofDouble := 8). > 1 to: values dataSize by: sizeofDouble do: > [:i| values doubleAt: i put: rand next]. > orig := (1 to: values dataSize by: sizeofDouble) collect: [:i|
values
> doubleAt: i]. > cb := Callback > signature: #(int (*)(const void *, const void *)) > block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1))
sign].
> (Alien lookup: 'qsort' inLibrary: Alien libcName) > primFFICallResult: nil > with: values pointer > with: nElements > with: sizeofDouble > with: cb thunk. > sort := (1 to: values dataSize by: sizeofDouble) collect: [:i|
values
> doubleAt: i]. > values free. > ^orig -> sort > > The above example uses Alien to make the callout. To use it with
the
> FFI simply pass cb pointer as the argument as is done above. > > > Implementation: > > The way that it works is in two parts > - the VM passes up a pointer to a structure from which all
arguments,
> stacked and in registers (because the VM has copied the register
args
> into the struct) can be accessed, and through which the result can
be
> returned. > - the image level provides marshalling methods that match the > signature in the callback > > So e.g. with a callback of > Callback > signature: #(int (*)(const void *, const void *)) > block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1))
sign]
> the marshalling methods are in Callback's signature protocol: > > Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien > <signature: #(int (*)(const void *, const void *)) abi: 'IA32'> > ^callbackContext wordResult: > (block > value: (Alien forPointer: (spAlien unsignedLongAt: 1)) > value: (Alien forPointer: (spAlien unsignedLongAt: 5))) > > where spAlien is an Alien pointing to a VMCallbackContext32. > > For ARM support we would add > > Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien > intRegArgs: regsAlien > <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'> > ^callbackContext wordResult: > (block > value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) > value: (Alien forPointer: (regsAlien unsignedLongAt: 5))) > > Basically the idea is that the selector of the method doesn't matter > except for the number of arguments. What's important is the pragma > which defines the signature and the ABI for which this is a valid > marshalling method. > > When the callback is instantiated, Callback introspects to find the > marshalling method that matches the signature. If one doesn't
already
> exist you can write one. Hopefully we'll write an ABI compiler that > will automatically generate these marshalling methods according to
the
> platform's ABI, but for now its a manual process. But at least it's > open and flexible. When the callback is invoked the evaluator is > performed with the current callbackContext and pointer(s) to the > arguments. There is a 32-bit and a 64-bit callback context, and it
can
> have a stack pointer, integer register args and floating point > register args. So it's general enough for any callback. > > To pass back the result, a value is assigned into the struct via the > accessor in the marshalling method and control returns to teh point > where teh callback comes in, and this uses a primitive to return. > Inside the callbackCOntext is a jmpbuf from a setjmp. The primitive > longjmp's back to the entry point in the VM which extracts the
result
> and the code for the kind of result and returns: > > Callback class>>invokeCallbackContext: vmCallbackContextAddress > "<Integer>" "^<FFICallbackReturnValue>" > "The low-level entry-point for callbacks sent from the VM/IA32ABI > plugin. > Return via primReturnFromContext:through:. thisContext's sender is
the
> call-out context." > | callbackAlien type | > callbackAlien := (Smalltalk wordSize = 4 > ifTrue: [VMCallbackContext32] > ifFalse: [VMCallbackContext64]) > atAddress: vmCallbackContextAddress. > [type := Callback evaluateCallbackForContext: callbackAlien] > ifCurtailed: [self error: 'attempt to non-local return across a > callback']. > type ifNil: > [type := 1. callbackAlien wordResult: -1]. > callbackAlien primReturnAs: type fromContext: thisContext > > On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig@netjam.org > mailto:craig@netjam.org> wrote: > > > Hi all-- > > I'd like to use a C shared library via FFI from Squeak or > Pharo. > Specifically, I want to call a C function which takes as one of
its
> parameters a pointer to an array of callback function addresses. > Does > one of the FFI approaches floating around handle that, and
provide a
> relatively pleasant mapping between Smalltalk block closures and
C
> callback function addresses? > > I was doing this so far in GemStone, but apparently its FFI > only > supports passing callback function addresses one at a time as
direct
> parameters of C callouts. I suppose I could write a wrapper C > library > around the one I really want to use, that provides the callback > setup > interface that GemStone expects, but whenever I have to write
actual
> C > code I start to think "Hm, why don't I just use a Smalltalk VM
for
> which > I have the source?" :) All the fancy distributed object-database > stuff > that my project also wants can wait for a bit. > > > thanks! > > -C > > -- > Craig Latta > netjam.org http://netjam.org > +31 6 2757 7177 tel:%2B31%20%20%206%202757%207177 (SMS ok) > + 1 415 287 3547 tel:%2B%201%20415%20%20287%203547 (no SMS) > > > > > > -- > _,,,^..^,,,_ > best, Eliot
I'd agree that the best thing for now is to leave it as an open question and then we should go for a solution that everyone feels comfortable with.
I'm not sure though if a single GitHub repository will do the trick considering the meta information mentioned by David. However, we could e.g. use different branches for different Squeak versions, but we really need to think this through.
A related question: how hard would it be to build plugins automatically (e.g. on Travis CI or on AppVeyor) from source?
What about using new SqueakMap entries to find the plugins?
The plugins may be in many places.
--Hannes
On 9/5/15, Fabio Niephaus lists@fniephaus.com wrote:
On Fri, Sep 4, 2015 at 9:26 AM Frank Shearar frank.shearar@gmail.com wrote:
On 4 September 2015 at 07:01, Tobias Pape Das.Linux@gmx.de wrote:
On 04.09.2015, at 05:46, Chris Muller asqueaker@gmail.com wrote:
I do think that anything is better than the current situation. A
bucket of
any kind in an agreed upon location seems like a good place to start.
I totally agree!
what about
http://github.com/squeak-vm/plugins
or
My preferred choice: the organisation already exists.
frank
Best -tobias
PS: we should look at https://github.com/search?q=squeak+vm
I hadn't considered that plugins would need meta data or labels.
They could be put into a .zip file and stored and cataloged somewhere which would be a well-known place for Squeak. And, each plugin would be tagged with one or more hierarchical Category's to indicate the platform and whatever else we wanted...
And, if the plugin is compiled and hosted somewhere else, like GitHub, then the Squeak-well-known place should direct the user to THAT external place instead of hosting copies on squeak.org servers... Except as a backup, of course..
Hmm, if only we have something that could do all that... Oh wait, we do!!!! ;-)
Perhaps the best thing is to leave it as an open question for a few
weeks
before settling on any kind of solution. A common location for plugins
is
about as far as my thinking takes me at the moment. I'm sure the
collective
imagination of the community can come up with a better idea.
Chris
On 2015-09-03 10:07 PM, David T. Lewis wrote:
On Thu, Sep 03, 2015 at 09:05:07PM -0400, Chris Cunnington wrote: > > This community needs a repository for plugins. A directory at > http://ftp.squeak.org ought to do it. > > If I want to try Alien I need the plugin. Where am I going to get
that?
> I could get VMMaker, read NewSqueakIA32ABIPlugin three times to see
it
> is not saying Newspeak and realize it compiles a plugin called
IA32ABI,
> which - coincidentally - is the name of the plugin the error Alien > exampleCqsort will gives me. (NewSqueakIA32ABIPlugin class>>
moduleName
> returns IA32ABI). > > Does anybody know what the Genie plugin does? Does Paul DeBrucker
need
> to store a BerkeleyDB plugin dll he got by emailing somebody on
GitHub?
> > If I understand this correctly, if such a directory existed, then
all a
> person would need to do to try the above Alien code would be to
download
> a pre-compiled plugin from - say - > http://ftp.squeak/plugins/Alien/fooVersion, put it in their VM
directory
> and start the image. Then check with SmalltalkImage current > listLoadedModules to see if you're ready to go. > > I nominate Fabio Niephaus to be the person the community can send > pre-compiled plugins for inclusion in such a directory. Eliot's > been > banging away about Alien for some time [1]. If a person could go, > get > the plugin and give the code shot, he could have more productive > conversations about Alien. > > So, if, Eliot, you have a pre-compiled plugin for Alien, perhaps in
an
> idle moment, you could email it to Fabio? And when, Fabio, you've > created the directory in http://ftp.squeak.org you could announce > it > here on Squeak-dev? > > FWIW, > > Chris
A word of caution:
I think the idea is good in principle, but it needs to be accompanied by a commitment by someone to do the hard work of keeping track of dependencies on the various flavors of OS, image word size, VM word
size,
spur/V3, and so forth. If someone would like to take that on, great. That said, we don't want to start a project that gets 80% completed,
and
then expect "the community" to do the rest of the work.
I cannot speak for Fabio, but I don't think it is fair for us to
expect
him to take on this kind of workload unless he has indicated an
interest
in doing it.
Maybe a restricted repository focused just on compiled plugins known to work with the Spur VM on a limited range of OS flavors would be
helpful
at this stage. It probably does not belong on ftp.squeak.org, but it
could
be a very useful resource if someone (Chris Cunnington?) wants to set
it
up.
Dave
> [1] http://wiki.squeak.org/squeak/uploads/6100/ > > On 2015-09-03 7:07 PM, Eliot Miranda wrote: >> >> Hi Craig, >> >> you need Alien-eem.24 >> >> Here's Alien class>>exampleCqsort >> "Call the libc qsort function (which requires a callback)." >> "Alien exampleCqsort" >> "(Time millisecondsToRun: [100 timesRepeat: [Alien >> exampleCqsort]])
/
>> 100.0" >> | cb rand nElements sizeofDouble values orig sort | >> rand := Random new. >> values := Alien newC: (nElements := 100) * (sizeofDouble := 8). >> 1 to: values dataSize by: sizeofDouble do: >> [:i| values doubleAt: i put: rand next]. >> orig := (1 to: values dataSize by: sizeofDouble) collect: [:i|
values
>> doubleAt: i]. >> cb := Callback >> signature: #(int (*)(const void *, const void *)) >> block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1))
sign].
>> (Alien lookup: 'qsort' inLibrary: Alien libcName) >> primFFICallResult: nil >> with: values pointer >> with: nElements >> with: sizeofDouble >> with: cb thunk. >> sort := (1 to: values dataSize by: sizeofDouble) collect: [:i|
values
>> doubleAt: i]. >> values free. >> ^orig -> sort >> >> The above example uses Alien to make the callout. To use it with
the
>> FFI simply pass cb pointer as the argument as is done above. >> >> >> Implementation: >> >> The way that it works is in two parts >> - the VM passes up a pointer to a structure from which all
arguments,
>> stacked and in registers (because the VM has copied the register
args
>> into the struct) can be accessed, and through which the result can
be
>> returned. >> - the image level provides marshalling methods that match the >> signature in the callback >> >> So e.g. with a callback of >> Callback >> signature: #(int (*)(const void *, const void *)) >> block: [ :arg1 :arg2 | ((arg1 doubleAt: 1) - (arg2 doubleAt: 1))
sign]
>> the marshalling methods are in Callback's signature protocol: >> >> Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien >> <signature: #(int (*)(const void *, const void *)) abi: 'IA32'> >> ^callbackContext wordResult: >> (block >> value: (Alien forPointer: (spAlien unsignedLongAt: 1)) >> value: (Alien forPointer: (spAlien unsignedLongAt: 5))) >> >> where spAlien is an Alien pointing to a VMCallbackContext32. >> >> For ARM support we would add >> >> Callback>>voidstarvoidstarRetint: callbackContext sp: spAlien >> intRegArgs: regsAlien >> <signature: #(int (*)(const void *, const void *)) abi: 'ARMV5'> >> ^callbackContext wordResult: >> (block >> value: (Alien forPointer: (regsAlien unsignedLongAt: 1)) >> value: (Alien forPointer: (regsAlien unsignedLongAt: 5))) >> >> Basically the idea is that the selector of the method doesn't >> matter >> except for the number of arguments. What's important is the >> pragma >> which defines the signature and the ABI for which this is a valid >> marshalling method. >> >> When the callback is instantiated, Callback introspects to find >> the >> marshalling method that matches the signature. If one doesn't
already
>> exist you can write one. Hopefully we'll write an ABI compiler >> that >> will automatically generate these marshalling methods according to
the
>> platform's ABI, but for now its a manual process. But at least >> it's >> open and flexible. When the callback is invoked the evaluator is >> performed with the current callbackContext and pointer(s) to the >> arguments. There is a 32-bit and a 64-bit callback context, and it
can
>> have a stack pointer, integer register args and floating point >> register args. So it's general enough for any callback. >> >> To pass back the result, a value is assigned into the struct via >> the >> accessor in the marshalling method and control returns to teh >> point >> where teh callback comes in, and this uses a primitive to return. >> Inside the callbackCOntext is a jmpbuf from a setjmp. The >> primitive >> longjmp's back to the entry point in the VM which extracts the
result
>> and the code for the kind of result and returns: >> >> Callback class>>invokeCallbackContext: vmCallbackContextAddress >> "<Integer>" "^<FFICallbackReturnValue>" >> "The low-level entry-point for callbacks sent from the VM/IA32ABI >> plugin. >> Return via primReturnFromContext:through:. thisContext's sender >> is
the
>> call-out context." >> | callbackAlien type | >> callbackAlien := (Smalltalk wordSize = 4 >> ifTrue: [VMCallbackContext32] >> ifFalse: [VMCallbackContext64]) >> atAddress: vmCallbackContextAddress. >> [type := Callback evaluateCallbackForContext: callbackAlien] >> ifCurtailed: [self error: 'attempt to non-local return across a >> callback']. >> type ifNil: >> [type := 1. callbackAlien wordResult: -1]. >> callbackAlien primReturnAs: type fromContext: thisContext >> >> On Thu, Sep 3, 2015 at 5:22 AM, Craig Latta <craig@netjam.org >> mailto:craig@netjam.org> wrote: >> >> >> Hi all-- >> >> I'd like to use a C shared library via FFI from Squeak or >> Pharo. >> Specifically, I want to call a C function which takes as one of
its
>> parameters a pointer to an array of callback function >> addresses. >> Does >> one of the FFI approaches floating around handle that, and
provide a
>> relatively pleasant mapping between Smalltalk block closures >> and
C
>> callback function addresses? >> >> I was doing this so far in GemStone, but apparently its >> FFI >> only >> supports passing callback function addresses one at a time as
direct
>> parameters of C callouts. I suppose I could write a wrapper C >> library >> around the one I really want to use, that provides the callback >> setup >> interface that GemStone expects, but whenever I have to write
actual
>> C >> code I start to think "Hm, why don't I just use a Smalltalk VM
for
>> which >> I have the source?" :) All the fancy distributed >> object-database >> stuff >> that my project also wants can wait for a bit. >> >> >> thanks! >> >> -C >> >> -- >> Craig Latta >> netjam.org http://netjam.org >> +31 6 2757 7177 tel:%2B31%20%20%206%202757%207177 (SMS ok) >> + 1 415 287 3547 tel:%2B%201%20415%20%20287%203547 (no SMS) >> >> >> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot
I'd agree that the best thing for now is to leave it as an open question and then we should go for a solution that everyone feels comfortable with.
I'm not sure though if a single GitHub repository will do the trick considering the meta information mentioned by David. However, we could e.g. use different branches for different Squeak versions, but we really need to think this through.
A related question: how hard would it be to build plugins automatically (e.g. on Travis CI or on AppVeyor) from source?
On 05.09.2015, at 07:31, H. Hirzel hannes.hirzel@gmail.com wrote:
What about using new SqueakMap entries to find the plugins?
The plugins may be in many places.
This won't help for the binaries, tho.
Best regards -Tobias
I mean to use the function of SqueakMap being a catalogue which references artefacts available elsewhere.
For dealing with binaries additional provision has to be made to deal with it.
--Hannes
On 9/5/15, Tobias Pape Das.Linux@gmx.de wrote:
On 05.09.2015, at 07:31, H. Hirzel hannes.hirzel@gmail.com wrote:
What about using new SqueakMap entries to find the plugins?
The plugins may be in many places.
This won't help for the binaries, tho.
Best regards -Tobias
Or better a repository here
https://github.com/squeak-smalltalk
On 9/5/15, H. Hirzel hannes.hirzel@gmail.com wrote:
I mean to use the function of SqueakMap being a catalogue which references artefacts available elsewhere.
For dealing with binaries additional provision has to be made to deal with it.
--Hannes
On 9/5/15, Tobias Pape Das.Linux@gmx.de wrote:
On 05.09.2015, at 07:31, H. Hirzel hannes.hirzel@gmail.com wrote:
What about using new SqueakMap entries to find the plugins?
The plugins may be in many places.
This won't help for the binaries, tho.
Best regards -Tobias
On Sat, Sep 5, 2015 at 2:19 AM, H. Hirzel hannes.hirzel@gmail.com wrote:
I mean to use the function of SqueakMap being a catalogue which references artefacts available elsewhere.
Yep, binaries are no problem for SqueakMap, it was designed to solve exactly this problem -- the cataloging of any software, in any format, hosted at any location.
For dealing with binaries additional provision has to be made to deal with it.
No additional provisions need to be made, except possibly adding a "Plugin" category.
On 05.09.2015, at 19:16, Chris Muller asqueaker@gmail.com wrote:
On Sat, Sep 5, 2015 at 2:19 AM, H. Hirzel hannes.hirzel@gmail.com wrote:
I mean to use the function of SqueakMap being a catalogue which references artefacts available elsewhere.
Yep, binaries are no problem for SqueakMap, it was designed to solve exactly this problem -- the cataloging of any software, in any format, hosted at any location.
For dealing with binaries additional provision has to be made to deal with it.
No additional provisions need to be made, except possibly adding a "Plugin" category.
I'm unsure. Its external things, nothing that is in image.
In my humble option, the 'execute anything' approach of SqueakMap has its appeal, but I don't think its sustainable. I'd like to keep existing stuff there, but I'd refrain from putting new projects there, or only if SqueakMap would support Metacello natively.
Best regards -Tobias
On Sat, Sep 5, 2015 at 2:19 AM, H. Hirzel hannes.hirzel@gmail.com wrote:
I mean to use the function of SqueakMap being a catalogue which references artefacts available elsewhere.
Yep, binaries are no problem for SqueakMap, it was designed to solve exactly this problem -- the cataloging of any software, in any format, hosted at any location.
For dealing with binaries additional provision has to be made to deal with it.
No additional provisions need to be made, except possibly adding a "Plugin" category.
I'm unsure. Its external things, nothing that is in image.
The image knows where the VM is installed. With SqueakMap you could browse all entries in the new "Plugin" Category, and simply right-click "Install" any of them straight into the system on the spot, without ever leaving the image. SqueakMap automatically backs things up in its local cache, so there'd be a copy there too..
In my humble option, the 'execute anything' approach of SqueakMap has its appeal, but I don't think its sustainable.
Actually, sustainability and avoiding bit-rot were major core requirements [1] of the 2011 effort, and, in fact, having a catalog of pure Smalltalk snippets organized and hierarchically tagged, is of the aspects that has contributed to its sustainability. If you are going to say something is unsustainable, please say why..
I'd like to keep existing stuff there,
Right, and that's why "existing stuff" remains still accessible to this day, and thru tommorrow, because it this approach _is_ sustainable. If it weren't, they wouldn't be..
but I'd refrain from putting new projects there, or only if SqueakMap would support Metacello natively.
I really don't understand why you feel threatened by SqueakMap -- or why you feel it is a substitute or alternative to Metacello. It's just documentation for goodness' sake. Is it okay with you if some of us document some project locations to help newbies and ourselves?
What about that new user just last week? He wanted to install Seaside into Squeak 5. Now, if seaside.st or squeak.org/projects would just refer users exclusively to SqueakMap for installation, then he would have been playing with Seaside in under 5 minutes because, guess what, Seaside 3.1.1 installs flawlessly into Squeak 5 from SqueakMap right now.
Unfortunately, seaside.st and squeak.org/projects decides instead to provide the little snippets of code for installation right there on the web-page (hard-coded? how is that for sustainable?) which, surprise surprise, *don't work*. Another user, left high-and-dry. Cases like his bother me, and are what caused me to expend all that effort in 2011 to solve this issue. If there's something wrong with it, could you help me fix it instead of tearing it down?
I think most people like Squeak to have an "App Store", and want their projects to be visible there, that's why we are already up to 23 packages in there for 5.0. If making SqueakMap support Metacello natively would win your support, it would make me very happy. It already supports about 5 or 10 other formats natively, why not?
- Chris
Hi all--
I loaded Alien-eem.25 into Squeak 5.0.15110 on Mac OS 10.11.15A279b from the Monticello HTTP repository at [1]. I ran AlienSunit in the Test Runner, and it Squeak. Before I get into posting error logs, can anyone confirm that AlienSunit runs successfully on their machine, on any host platform, or that I shouldn't expect it to run successfully? Are there other Monticello packages I should be loading, and if so, in what order?
thanks!
-C
[1] http://www.squeaksource.com/Alien
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Hi Craig,
On Mon, Sep 14, 2015 at 5:00 PM, Craig Latta craig@netjam.org wrote:
Hi all--
I loaded Alien-eem.25 into Squeak 5.0.15110 on Mac OS 10.11.15A279bfrom the Monticello HTTP repository at [1]. I ran AlienSunit in the Test Runner, and it Squeak. Before I get into posting error logs, can anyone confirm that AlienSunit runs successfully on their machine, on any host platform, or that I shouldn't expect it to run successfully? Are there other Monticello packages I should be loading, and if so, in what order?
I get 15 passes and one failure, that being testLongLong. But that's in my VMMaker image, which has some extras in it. AFAIA tough Alien should just work. I'll look at the testLongLong failure soon cuz I'm starting on the x64 JIT and that requires the above to work to access the processor alien.
thanks!-C
[1] http://www.squeaksource.com/Alien
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok)
- 1 415 287 3547 (no SMS)
Hi Eliot--
I loaded Alien-eem.25 into Squeak 5.0.15110 on Mac OS 10.11.15A279b from the Monticello HTTP repository at [1]. I ran AlienSunit in the Test Runner, and it crashed Squeak. Before I get into posting error logs, can anyone confirm that AlienSunit runs successfully on their machine, on any host platform, or that I shouldn't expect it to run successfully? Are there other Monticello packages I should be loading, and if so, in what order?
I get 15 passes and one failure, that being testLongLong. But that's in my VMMaker image, which has some extras in it. AFAIA tough Alien should just work. I'll look at the testLongLong failure soon cuz I'm starting on the x64 JIT and that requires the above to work to access the processor alien.
Would you please try with the current Squeak 5 release?
Also, is it feasible to make FFI calls to a 64-bit C library?
thanks again!
-C
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
On Tue, Sep 15, 2015 at 02:34:52AM +0200, Craig Latta wrote:
Also, is it feasible to make FFI calls to a 64-bit C library?
I got 64-bit FFI working about five years ago but we never got it merged into the VM (sorry). See http://bugs.squeak.org/view.php?id=7237
Those patches would be pretty badly bit-rotted by now, so I'm not sure of the current state of affairs. I would anticipate a few issues, because getting it to work in the old FFI was not trivial. It's definitely worth getting this working again, but it may take a bit of work to make it happen. Maybe it's easier with Eliot's more recent work, because I think the assembler glue goes away and that may simplify matters.
Dave
I get a lock up and a crash with the all-in-one. No problem with the latest VM. No diagnosis as yet.
On Mon, Sep 14, 2015 at 5:34 PM, Craig Latta craig@netjam.org wrote:
Hi Eliot--
I loaded Alien-eem.25 into Squeak 5.0.15110 on Mac OS 10.11.15A279b from the Monticello HTTP repository at [1]. I ran AlienSunit in the Test Runner, and it crashed Squeak. Before I get into posting error logs, can anyone confirm that AlienSunit runs successfully on their machine, on any host platform, or that I shouldn't expect it to run successfully? Are there other Monticello packages I should be loading, and if so, in what order?
I get 15 passes and one failure, that being testLongLong. But that's in my VMMaker image, which has some extras in it. AFAIA tough Alien should just work. I'll look at the testLongLong failure soon cuz I'm starting on the x64 JIT and that requires the above to work to access the processor alien.
Would you please try with the current Squeak 5 release? Also, is it feasible to make FFI calls to a 64-bit C library? thanks again!-C
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok)
- 1 415 287 3547 (no SMS)
_,,,^..^,,,_ best, Eliot
Hi Craig,
On Mon, Sep 14, 2015 at 5:34 PM, Craig Latta craig@netjam.org wrote:
Hi Eliot--
I loaded Alien-eem.25 into Squeak 5.0.15110 on Mac OS 10.11.15A279b from the Monticello HTTP repository at [1]. I ran AlienSunit in the Test Runner, and it crashed Squeak. Before I get into posting error logs, can anyone confirm that AlienSunit runs successfully on their machine, on any host platform, or that I shouldn't expect it to run successfully? Are there other Monticello packages I should be loading, and if so, in what order?
I get 15 passes and one failure, that being testLongLong. But that's in my VMMaker image, which has some extras in it. AFAIA tough Alien should just work. I'll look at the testLongLong failure soon cuz I'm starting on the x64 JIT and that requires the above to work to access the processor alien.
Would you please try with the current Squeak 5 release?
OK, so in an updated Squeak 5 release containing only Alien
- the 5.0 All-in-one locks up and core dumps - the same VM from my site (CogSpur.app-15.27.3397.tgz http://www.mirandabanda.org/files/Cog/VM/VM.r3397/CogSpur.app-15.27.3397.tgz) fails the testLongLong test but all others pass.
So there's something strange about the all-in-one. I'll try and investigate further as time allows but perhaps you could look at the plugins and see whether one of them is rogue, e.g. by removing the external plugins one by one?
Also, is it feasible to make FFI calls to a 64-bit C library?
Not from a 32-bit executable. One can't even load a 64-bit library into a 32-bit executable. Building a 64-bit version of the 32-bit VM is possible but a lot of work (needs a 674-nit JIT). Instead I'm pursuing a proper all-64-bit solution.
thanks again!-C
-- Craig Latta netjam.org +31 6 2757 7177 (SMS ok)
- 1 415 287 3547 (no SMS)
squeak-dev@lists.squeakfoundation.org
-
Chris Cunnington -
Chris Muller -
Chris Muller -
Craig Latta -
David T. Lewis -
Eliot Miranda -
Fabio Niephaus -
Frank Shearar -
H. Hirzel -
Tobias Pape