On Fri, Jul 22, 2011 at 9:08 AM, Ricardo Moran richi.moran@gmail.comwrote:
Yes, indeed. Right now, for most use cases we use the SerialPort class, but we still have some FFI code for other stuff hanging around. I plan to write it as a plugin as soon as I have the time.
But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? We can always add security to the FFI (essentially some way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). So I really do appeal to people to help me put the effort in here. I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. I /can/ work on the threaded FFI as we need it at Cadence, just as many other people need it. But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need help on the platforms I can't justify working on.
So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. Let's defer gratification :)
Cheers, RIcho
On Fri, Jul 22, 2011 at 10:37 AM, Bert Freudenberg bert@freudenbergs.dewrote:
It's not part of Etoys yet. If its low-level interface indeed requires FFI, then that interface would have to be rewritten as plugin for Etoys to include it.
- Bert -
On 22.07.2011, at 14:55, karl ramberg wrote:
What about the Physical Etoys project ? http://tecnodacta.com.ar/gira/projects/physical-etoys/
Karl
On Fri, Jul 22, 2011 at 2:20 PM, Bert Freudenberg bert@freudenbergs.dewrote:
On 22.07.2011, at 02:10, Eliot Miranda wrote:
On Thu, Jul 21, 2011 at 4:57 PM, Igor Stasenko siguctua@gmail.com
wrote:
On 22 July 2011 02:34, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All, today I committed the threaded FFI plugin along with the threaded
VM and
hope to be able to support people in doing threaded FFI work on the
x86
platforms really soon now (Esteban, amongst others, this means you ;)
).
But to put the image-level code that uses the new FFI plugin into production the standard VM needs to be upgraded to not crash with the ExternalFunction image changes the new threaded FFI plugin wants.
The main
change is that ExternalFunction changes size, gaining a stackSize
inst var
which is used to tell the FFI plugin how much stack spacer to reserve
for
marshalling (its computed on first use). The best way for this to
happen is
to start using the ThreadedFFIPlugin as the standard plugin in the Interpreter VM. The ThreadedFFIPlugin overall is simpler than the old
FFI
plugin (no assembler support code) and better (fully reentrant). But
for
this to happen we may need to implement the platform=specific parts
of
the ThreadedFFIPlugin (still entirely Smalltalk code) for PowerPC and
ARM
(and perhaps others, SPARC??). I understand what has to be done and
I can
explain it to anyone interested in working on the problem, but I
don't have
any hardware with which I can test the results so its pretty
pointless my
doing the work and producing something inevitably broken. So are
there
people out there who would be interested in doing the port to PPC and
ARM?
It means fleshing out the classes ThreadedARMFFIPlugin ThreadedFFICalloutStateForARM ThreadedPPCBEFFIPlugin ThreadedFFICalloutStateForPPC and having a solid understanding of how alloca works to do stack
allocation
and how the ABIs on ARM and PPC pass registers. here's hoping there are a few good low-level people who would be
interested.
Let me know.
I wonder if you find anyone today using these archs, not saying using squeak/pharo on them + knowing low-level details + wanting to help :)
Right, chances are thin. But then if no one is interested then the
work doesn't need to be done. Bert, you mentioned the other day that PPC was still relevant for etoys. But am I right in thinking etoys never uses the FFI?
Right. Etoys should run on both ARM and PPC, but it does not use FFI.
- Bert -
On Jul 22, 2011, at 6:25 PM, Eliot Miranda wrote:
On Fri, Jul 22, 2011 at 9:08 AM, Ricardo Moran richi.moran@gmail.com wrote: Yes, indeed. Right now, for most use cases we use the SerialPort class, but we still have some FFI code for other stuff hanging around. I plan to write it as a plugin as soon as I have the time.
But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? We can always add security to the FFI (essentially some way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). So I really do appeal to people to help me put the effort in here. I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. I /can/ work on the threaded FFI as we need it at Cadence, just as many other people need it. But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need help on the platforms I can't justify working on.
So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. Let's defer gratification :)
Personally we would love but to do that you need to know. This is the key question behind increasing the mass of people knowing. This is why we organize the School and why the blog of mariano is good or the effort of igor to make everybody been able to compile the vm. It is a slow and long process. There is a difference between will and can. At least to me there are plenty of things I would like to do but I'm quite sure I will not be able to do them in this life.
Stef
On Fri, Jul 22, 2011 at 1:25 PM, Eliot Miranda eliot.miranda@gmail.comwrote:
On Fri, Jul 22, 2011 at 9:08 AM, Ricardo Moran richi.moran@gmail.comwrote:
Yes, indeed. Right now, for most use cases we use the SerialPort class, but we still have some FFI code for other stuff hanging around. I plan to write it as a plugin as soon as I have the time.
But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? We can always add security to the FFI (essentially some way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). So I really do appeal to people to help me put the effort in here. I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. I /can/ work on the threaded FFI as we need it at Cadence, just as many other people need it. But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need help on the platforms I can't justify working on.
So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. Let's defer gratification :)
Hi Eliot,
I would love to help but I don't have the hardware nor the knowledge to do it. Sorry...
Cheers, Richo
Cheers, RIcho
On Fri, Jul 22, 2011 at 10:37 AM, Bert Freudenberg bert@freudenbergs.dewrote:
It's not part of Etoys yet. If its low-level interface indeed requires FFI, then that interface would have to be rewritten as plugin for Etoys to include it.
- Bert -
On 22.07.2011, at 14:55, karl ramberg wrote:
What about the Physical Etoys project ? http://tecnodacta.com.ar/gira/projects/physical-etoys/
Karl
On Fri, Jul 22, 2011 at 2:20 PM, Bert Freudenberg bert@freudenbergs.dewrote:
On 22.07.2011, at 02:10, Eliot Miranda wrote:
On Thu, Jul 21, 2011 at 4:57 PM, Igor Stasenko siguctua@gmail.com
wrote:
On 22 July 2011 02:34, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All, today I committed the threaded FFI plugin along with the
threaded VM and
hope to be able to support people in doing threaded FFI work on the
x86
platforms really soon now (Esteban, amongst others, this means you
;) ).
But to put the image-level code that uses the new FFI plugin into production the standard VM needs to be upgraded to not crash with
the
ExternalFunction image changes the new threaded FFI plugin wants.
The main
change is that ExternalFunction changes size, gaining a stackSize
inst var
which is used to tell the FFI plugin how much stack spacer to
reserve for
marshalling (its computed on first use). The best way for this to
happen is
to start using the ThreadedFFIPlugin as the standard plugin in the Interpreter VM. The ThreadedFFIPlugin overall is simpler than the
old FFI
plugin (no assembler support code) and better (fully reentrant).
But for
this to happen we may need to implement the platform=specific parts
of
the ThreadedFFIPlugin (still entirely Smalltalk code) for PowerPC
and ARM
(and perhaps others, SPARC??). I understand what has to be done and
I can
explain it to anyone interested in working on the problem, but I
don't have
any hardware with which I can test the results so its pretty
pointless my
doing the work and producing something inevitably broken. So are
there
people out there who would be interested in doing the port to PPC
and ARM?
It means fleshing out the classes ThreadedARMFFIPlugin ThreadedFFICalloutStateForARM ThreadedPPCBEFFIPlugin ThreadedFFICalloutStateForPPC and having a solid understanding of how alloca works to do stack
allocation
and how the ABIs on ARM and PPC pass registers. here's hoping there are a few good low-level people who would be
interested.
Let me know.
I wonder if you find anyone today using these archs, not saying using squeak/pharo on them + knowing low-level details + wanting to help :)
Right, chances are thin. But then if no one is interested then the
work doesn't need to be done. Bert, you mentioned the other day that PPC was still relevant for etoys. But am I right in thinking etoys never uses the FFI?
Right. Etoys should run on both ARM and PPC, but it does not use FFI.
- Bert -
-- best, Eliot
On Fri, 22 Jul 2011, Eliot Miranda wrote:
But getting the FFI working on ARM and/or PPC is more effective than writing yet another plugin, no? We can always add security to the FFI (essentially some way of disabling internal calls within the FFI plugin, e.g. based on cryptographic keys to enable it, or whatever). So I really do appeal to people to help me put the effort in here. I can't justify working on the FFI on ARM or PPC in my Cadence Newspeak gig; we're entirely on x86. I /can/ work on the threaded FFI as we need it at Cadence, just as many other people need it. But for me to move the threaded FFI ahead it needs to be on all platforms and that means I need help on the platforms I can't justify working on.
So folks, please give serious consideration to putting effort into something that may be a little harder at first but will yield major benefits slightly later. Let's defer gratification :)
QEMU can be used for emulating various CPUs, so if you don't have an ARM, PPC or SPARC hardware, you can still develop/test FFI with it.
Threaded FFI is great, it opens new possibilities, but nowadays the feature I miss the most is the ability to call functions in 64-bit (x86_64) libraries (from a 32-bit VM/image).
Levente
On Sat, Jul 23, 2011 at 01:50:49PM +0200, Levente Uzonyi wrote:
Threaded FFI is great, it opens new possibilities, but nowadays the feature I miss the most is the ability to call functions in 64-bit (x86_64) libraries (from a 32-bit VM/image).
As far as I know, mixing 32-bit and 64-bit libraries in a single executable (VM) is not feasible, at least not on Linux. Maybe it would work in the case of statically linked libraries, I'm not certain.
The traditional FFI does not work for 64-bit VMs, and I expect that this needs to be resolved for any new FFI implementation as well, because some of the issues were in the interpreter itself, not just the FFI plugin. The issue also may affect other things, e.g. Igor's proposed #primLoadModule primitive crashes the VM if compiled for 64 bit.
I had intended to push the fixes for 64-bit FFI as a VM team initiative, but it's a complex set of changes that require coordination of VMMaker (oscog plus traditional), as well as of all the platforms (oscog, trunk, all platforms) so IMO this would be a lot to take on right now.
But I guess that I should ask - is there interest in implementing fixes for 64-bit FFI?
Some background:
Mantis report and patches to implement the fixes, circa 2008. I assume some bit rot will have set in by this time: http://bugs.squeak.org/view.php?id=7237
Status of the fixes, as tested on various platforms: http://lists.squeakfoundation.org/pipermail/vm-dev/2008-May/001945.html
The last discussion relative to VM team priorities: http://lists.squeakfoundation.org/pipermail/vm-dev/2009-July/002853.html
Dave
vm-dev@lists.squeakfoundation.org
-
David T. Lewis -
Eliot Miranda -
Levente Uzonyi -
Ricardo Moran -
stephane ducasse