Hi Koh,

On Mon, Jul 27, 2009 at 3:01 PM, askoh askoh@askoh.com wrote:

We would like to make Squeak callable from other Windows programs. For example, a C# program calling squeak.dll. It can be headless for now. How can we change the dll-VM so that the interpreter executes commands passed from the calling C# program and returns an answer?

Here's my understanding of how to do it based on work I did for VisualWorks. This is an edited version of a message I sent to some people outsiode of Cincom so I think it's OK to reuse.

"This is quite hard, or merely non-trivial, depending on what you want to do. There appear to be three architectures, one single-threaded and two multi-threaded. In the single-threaded architecture the VM runs on the thread that calls-in and runs only until it answers a result to the call-in. In the multi-threaded architecture, the VM runs in its own thread(s) and each call-in is effectively a rendezvous between the calling thread and the VM thread(s).

At first blush the single-threaded architecture (STA1) appears quite simple and is potentially the fastest. But there are lots of hidden problems with it. One can only provide simple call-ins. One can't for example expect to run a server application on the VM if its only allowed to run until it answers a result. Basically the single-threaded architecture is only suitable for Smalltalk libraries that provide only computational entry-points, not server or glue functionality. This architecture is not thread-safe. Multiple client threads attempting to call-in to the VM will break the system as the VM is not natively multi-threaded in Smalltalk, only in the C connect.

In addition, to make the single-threaded architecture work a lot of low-level scheduling/I/O code in the VM has to be disabled and/or modified to ensure the system just responds to a call-in with a prompt result.

The first multi-threaded architecture (MTA1) has a much more complicated glue (as it involves a rendezvous) but is actually quite straight-forward given the threaded facilities in the C connect. In fact I did a prototype of MTA1 in 1997/1998 to implement a web-browser plugin.

In MTA1 the library is loaded with an initialization routine that e.g. specifies an image to load (alternatively the image is linked into the library as a resource, but in any case there will be an initalization call). The initialization call spawns a thread that becomes the VM's main thread and waits on a semaphore until the spawned thread has initialized Smalltalk. This thread first loads an image (either from a file or resource) and then once initialized signals the semaphore, allowing the intialization call to return. Form then on subsequent call-ins are made via a rendezvous between whatever threads call-in and the VM thread(s). These rendezvous are hundreds of times slower than a simple C call-in (still only a few microseconds on modern hardware) but one has a fully thread-safe soluton that can provide any functionality one may wish to export, including graphics. My web plugin prototype worked well.

[We didn't pursue it further since a) the VM library shares its address space with the rest of the browser, including other plugins and hence is potentialy vulnerable to heap corruption and hence creates a maintennance nightmare, and b) since graphics handles can be shared between processes on most graphics platforms (certainly windows and X11) one can use a separate process. So one can create a functional web plugin via a small glue DLL that gets loaded as a plugn, spawns a full VM process, handing it the graphics handle encoded via the command-line as a hex printString (!). The VM process creates a window around the passed-in handle and the OS takes care of outing events to the right place, etc, etc, etc].

But this suggests MTA2 where one again separates the VM address space from the client address space and uses some RPC-like mechanism running over a socket to communicate call-ins and their results from a glue DLL and the separate VM process. Performance should be quite similar. The wire encoding/decoding effort is small (and has to be done to some extent anyway to bridge the gap between whatever the client lanuage is and Smalltalk). The Smalltalk system spawns a Smalltalk process to read & decode call-ins from a pipe/socket, writing results once computed.

I'm convinced that STA1 is pretty pointless. MTA1 requires extensive work to the VM's call-back machinery. But MTA2 might provide acceptable performance with simpler machinery."

HTH

Thanks, Aik-Siong Koh -- View this message in context: http://www.nabble.com/step-through-VM-source-tp24678490p24689078.html Sent from the Squeak - Dev mailing list archive at Nabble.com.

On Thu, Aug 6, 2009 at 7:46 PM, askoh askoh@askoh.com wrote:

Thanks for the detailed article about making Squeak callable.

Instead of having a monolithic vm, can we break squeak.exe into squeak2.exe and squeak2.dll? The squeak2.exe will do the complex multi threaded tasks and call squeak2.dll which is more passive and simple. The novelty here is that squeak2.dll can be callable by any *.exe in any language. I also believe squeak2.dll would be useful for parallel processing in multi-cores. Is this a good idea? Is it doable?

Packaging as a split .exe/.dll is a prerequisite of being able to link to the VM and is trivial. The bulk of the work is in the asynchronous call-back machinery.

Thanks, Aik-Siong Koh -- View this message in context: http://www.nabble.com/step-through-VM-source-tp24678490p24857906.html Sent from the Squeak - Dev mailing list archive at Nabble.com.