<div dir="ltr"><div class="gmail_extra"><div>Hi All,</div><div><br></div><div> this is an important discussion that is taking a religious tone that we should strive to avoid. There are good arguments for plugins, namely security and encapsulation. There are good arguments for an FFI, namely extensibility and platform compatibility.</div>
<div><br></div><div>Plugins provide security because they allow the system to control any and all access to the underlying platform, permitting access only through plugins. With an FFI the underlying platform is exposed and one needs other mechanisms, for example Newspeak mirrors, to prevent untrusted code from accessing the platform with potentially disastrous effects (self shell: '/bin/rm -rf /*').</div>
<div><br></div><div>Plugins encapsulate all sorts of details behind a potentially simple primitive interface. This can avoid confusing the newcommer (but at the same time frustrate them by hiding details), provide portability, can make it easier to determine the extent of work in moving to a new OS platform, and so on.</div>
<div><br></div><div>An FFI allows immediate extensibility. External functionality can be invoked immediately. With plugins a primitive interface must be designed and then implemented. With the FFI the API is already defined; it must "merely" be accessed. This immediacy can itself provide simplicity, especially where callbacks and threads are involved. Plugins can hide a lot of complexity (e.g. the SocketPlugin encapsulates platform threads that are waiting on blocking calls so that Squeak itself is provided with an interrupt-driven interface, necessitated by the Squeak platform's lack of native thread support).</div>
<div><br></div><div>An FFI allows all underlying functionality to be accessed. The plugin approach necessitates defining a lowest common denominator approach to functionality, especially irksome in some applications where setting the right flag, e.g. on a socket stream, can have a significant performance impact.</div>
<div><br></div><div>So there are good arguments either way. In a system oriented towards safe play plugins make excellent sense. In a platform oriented towards industrial development an FFI is a must-have, and a weak one will really hurt acceptance.</div>
<div><br></div><div>IMO Squeak needs to have both. It needs plugins to provide its hallmarks such as eToys. But to be a more general platform it needs an FFI. Managing this split personality will take work but I don't see any fundamental issues. Having a well-factored base into which packages can be loaded to create different personalities is key, and good work is being done here. There may be a half-way house where the FFI is strictly encapsulated, but this is hypothetical. I know how to solve threads, pinning, etc, but I don't know off the top of my head how to encapsulate the FFI, so I can't propose it as a solution.</div>
<div><br></div><div>A number of straw men have been raised against the FFI in this discussion. OK, that's unfair. A number of important questions have been asked of the FFI in this discussion.</div><div><br></div><div>
Levente asks "<span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">Show me how you can replace the SocketPlugin with FFI, and I'll consider it. ;)".</span></div>
<div><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">The issue here is threads. The SocketPlugin encapsulates blocking calls, spawning hidden OS threads to make these calls and then signal semaphores when they complete. To solve this one needs both native thread support in the VM (and I have a prototype that needs Spur's facilities to make practicable) and pinning (the ability to stop certain objects moving). Spur provides pinning.</span></div>
<div><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif"><br></span></div><div><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">David says "</span><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">I remember when somebody on the Pharo list suggested reimplementing the</span></div>
<span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">OSProcessPlugin in FFI. I told them it was a really great idea, and they<br>should give it a try. That settled the matter quite quickly ;-)". Again they failed because of the lack of necessary underlying functionality from the VM. With threads, pinning and a way of expressing the array of pointers to strings idiom (a simple extension to marshalling, and/or pinning, e.g. provide an address of first field primitive) an FFI can do all the OSProcessPlugin can do and significantly simpler.</span></div>
<div class="gmail_extra"><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif"><br></span></div><div class="gmail_extra"><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">David also says "</span><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">it is a complete mystery to me why people are willing to </span><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">work so hard to avoid writing a VM plugin. VM plugins are reliable, </span><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">portable, and debuggable. They work across a range of processors. They </span><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">work on 64-bit platforms. So why would someone prefer to switch to a </span><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">calling interface that basically only works on 32-bit Intel processors </span><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">and that may require low level knowledge of calling conventions, word </span><span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">alignment, and platform-specific data types?"</span></div>
<div class="gmail_extra"><br></div><div class="gmail_extra">This is a non-sequitur. The sentences beginning "So why would someone..." don't follow from the first sentences. Writing the plugin requires even more knowledge than writing the FFI interface because one needs to know the VM facilities for mating Squeak objects to plugins. Writing plugins /and/ writing interfaces above FFIs are hard. But in my experience a powerful FFI provides a faster and easier development experience. Both can be difficult to port, but plugins have the advantage that only the innards have to be ported while facing the C code face. My experience in that regard leaves me with a preference for FFIs. The lack of a 64-bit FFI is a bad weakness of the Squeak platform, something Spur again makes easy to rectify.</div>
<div class="gmail_extra"><br></div><div class="gmail_extra">Bert asks "<span style="border-collapse:collapse;font-size:14px;font-family:arial,sans-serif">Suppose we add a new VM platform, like a VM running on JavaScript in the browser. Do you really want to re-implement all the C libraries utilized via FFI? Or rather a handful of primitives in your language of choice?</span>". First it is not clear that one *can* implement these primitives taking either approach. If the platform, e.g. JavaScript in a browser, takes the Squeak plugin approach of preventing access to the platform except through a restricted set of facilities, then certain functionality will simply be off-limits, whether one has an FFI or not. Second, reimplementing all the C libraries isn't obligatory. If the platform provides an FFI one simply mates to its FFI and accesses the underlying libraries. If it doesn't then that functionality is off-limits, but that doesn't mean the rest of the system doesn't work. It also means that Squeak running in that context is no less useful than any other platform, because the underlying platform (just as Squeak does with plugins)</div>
<div class="gmail_extra"><br></div><div class="gmail_extra">-- <br>best,<div>Eliot</div>
</div></div>