Hi All,

    this is an important discussion that is taking a religious tone that we
should strive to avoid.  There are good arguments for plugins, namely
security and encapsulation.  There are good arguments for an FFI, namely
extensibility and platform compatibility.

Plugins provide security because they allow the system to control any and
all access to the underlying platform, permitting access only through
plugins.  With an FFI the underlying platform is exposed and one needs
other mechanisms, for example Newspeak mirrors, to prevent untrusted code
from accessing the platform with potentially disastrous effects (self
shell: '/bin/rm -rf /*').

Plugins encapsulate all sorts of details behind a potentially simple
primitive interface.  This can avoid confusing the newcommer (but at the
same time frustrate them by hiding details), provide portability, can make
it easier to determine the extent of work in moving to a new OS platform,
and so on.

An FFI allows immediate extensibility.  External functionality can be
invoked immediately.  With plugins a primitive interface must be designed
and then implemented. With the FFI the API is already defined; it must
"merely" be accessed.  This immediacy can itself provide simplicity,
especially where callbacks and threads are involved.  Plugins can hide a
lot of complexity (e.g. the SocketPlugin encapsulates platform threads that
are waiting on blocking calls so that Squeak itself is provided with an
interrupt-driven interface, necessitated by the Squeak platform's lack of
native thread support).

An FFI allows all underlying functionality to be accessed.  The plugin
approach necessitates defining a lowest common denominator approach to
functionality, especially irksome in some applications where setting the
right flag, e.g. on a socket stream, can have a significant performance
impact.

So there are good arguments either way.  In a system oriented towards safe
play plugins make excellent sense.  In a platform oriented towards
industrial development an FFI is a must-have, and a weak one will really
hurt acceptance.

IMO Squeak needs to have both.  It needs plugins to provide its hallmarks
such as eToys.  But to be a more general platform it needs an FFI.
 Managing this split personality will take work but I don't see any
fundamental issues.  Having a well-factored base into which packages can be
loaded to create different personalities is key, and good work is being
done here.  There may be a half-way house where the FFI is strictly
encapsulated, but this is hypothetical.  I know how to solve threads,
pinning, etc, but I don't know off the top of my head how to encapsulate
the FFI, so I can't propose it as a solution.

A number of straw men have been raised against the FFI in this discussion.
 OK, that's unfair.  A number of important questions have been asked of the
FFI in this discussion.

Levente asks "Show me how you can replace the SocketPlugin with FFI, and
I'll consider it. ;)".
The issue here is threads.  The SocketPlugin encapsulates blocking calls,
spawning hidden OS threads to make these calls and then signal semaphores
when they complete.  To solve this one needs both native thread support in
the VM (and I have a prototype that needs Spur's facilities to make
practicable) and pinning (the ability to stop certain objects moving).
 Spur provides pinning.

David says "I remember when somebody on the Pharo list suggested
reimplementing the
OSProcessPlugin in FFI. I told them it was a really great idea, and they
should give it a try. That settled the matter quite quickly ;-)".  Again
they failed because of the lack of necessary underlying functionality from
the VM.  With threads, pinning and a way of expressing the array of
pointers to strings idiom (a simple extension to marshalling, and/or
pinning, e.g. provide an address of first field primitive) an FFI can do
all the OSProcessPlugin can do and significantly simpler.

David also says "it is a complete mystery to me why people are willing to work
so hard to avoid writing a VM plugin. VM plugins are reliable, portable,
and debuggable. They work across a range of processors. They work on 64-bit
platforms. So why would someone prefer to switch to a calling interface
that basically only works on 32-bit Intel processors and that may require
low level knowledge of calling conventions, word alignment, and
platform-specific data types?"

This is a non-sequitur.  The sentences beginning "So why would someone..."
don't follow from the first sentences.  Writing the plugin requires even
more knowledge than writing the FFI interface because one needs to know the
VM facilities for mating Squeak objects to plugins.  Writing plugins /and/
writing interfaces above FFIs are hard.  But in my experience a powerful
FFI provides a faster and easier development experience.  Both can be
difficult to port, but plugins have the advantage that only the innards
have to be ported while facing the C code face.  My experience in that
regard leaves me with a preference for FFIs.  The lack of a 64-bit FFI is a
bad weakness of the Squeak platform, something Spur again makes easy to
rectify.

Bert asks "Suppose we add a new VM platform, like a VM running on
JavaScript in the browser. Do you really want to re-implement all the C
libraries utilized via FFI? Or rather a handful of primitives in your
language of choice?".  First it is not clear that one *can* implement these
primitives taking either approach.  If the platform, e.g. JavaScript in a
browser, takes the Squeak plugin approach of preventing access to the
platform except through a restricted set of facilities, then certain
functionality will simply be off-limits, whether one has an FFI or not.
 Second, reimplementing all the C libraries isn't obligatory.  If the
platform provides an FFI one simply mates to its FFI and accesses the
underlying libraries.  If it doesn't then that functionality is off-limits,
but that doesn't mean the rest of the system doesn't work.  It also means
that Squeak running in that context is no less useful than any other
platform, because the underlying platform (just as Squeak does with plugins)

-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20131119/19051fa3/attachment.htm