[Vm-dev] Multi-core system Squeak (was Re: Sista alternate bytecodes + Java Bytecode ?)

[David T. Lewis]

On Fri, Oct 20, 2017 at 06:40:47AM +0800, Ben Coman wrote:
>  
> On Fri, Oct 20, 2017 at 12:46 AM, tim Rowledge <tim at rowledge.org> wrote:
> 
> >
> >
> > > On 19-10-2017, at 7:31 AM, Todd Blanchard <tblanchard at mac.com> wrote:
> > >
> > > It works very well and this idea of mini worker images sounds like it
> > might be a good fit for that model.
> > >
> >
> > One thing I noticed with surprise was how astonishingly fast Dave???s
> > OSProcess could spawn an entire image, even on a Pi. So it might be worth
> > experimenting with spawning a child image to do some processing and maybe
> > transmit results back via sockets or writing a project file or.. well,
> > whatever. You might think that on a tiny thing like a Pi you would rapidly
> > use up all memory and start disastrous paging to SD (ouch!) but a Pi has a
> > Gb or ram and my 6.0alpha development image uses less than 9% of that.
> >
> > It???s a potentially simple way to make some use of multiple cores.
> >
> 
> Are talking about booting up a new image?
> What about native-forking a running image.  Linux default copy-on-write
> (IIUC) should help limit memory usage, with full access to existing objects
> without needing to worry about multi-threaded access to objectspace.
> The trick to work out would the join mechanism. Perhaps returned objects
> would be limited to instance variables containing only immediate types and
> plain arrays of the same.  Maybe the join mechanism would need to groom out
> non-compliant objects at the VM level.  Or the join returns a STON
> representation.
> 
> cheers -ben

Yes that is exactly what Tim is talking about. The unix fork mechanism is
suprisingly efficient when applied to a VM process running a Smalltalk image. 
Fuel works nicely for returning objects to the parent image, although I'm sure
STON would work fine also.

The Spur image format allows object memory to be segmented such that it will
support fast and efficient forking of images even for extremely large images.
I don't have any real world use cases to prove the point, but would be quite
interested in finding some.

The join mechanism can be very simple. The child image serializes the result
of whatever work it has done, and writes it to a pipe connected to the parent
image. The parent deserializes the next object from the pipe, and that is the
result. As long as the pipe read does not block the VM (that is an OSProcess
thing), you can put any number of child readers into separate Smalltalk processes.

Background information is at http://wiki.squeak.org/squeak/6176.

Dave