On Sun, Nov 13, 2011 at 11:07:27AM -0600, Chris Muller wrote:
> Really cool Dave!  May I ask a couple of questions?
> 
> Is the initial object state of the forked image a clone of the forking
> image, so I have access to all of the variables and objects?  For
> example:
> 
>     RemoteTask
>         do: [ someObjectInThisImage calculateComplexValue ]
>         whenComplete: [ :calculatedValue | someObjectInThisImage
> complexValue: calculatedValue ]

Yes, that's right. The child image is an exact copy of the parent
except for the return value of the #forkSqueak call, which will
be 0 in the child image an a positive integer (the PID of the child)
in the parent "supervisor" image.
  
The #forkSqueak operation is similar to #snapshot:andQuit: in the
sense that the image "wakes up" after the snapshot resume point
(or after the fork) and decides what to do. In the case of
#snapshot:andQuit: the image that wakes up is identical to the
image that was saved. In the case of #forkSqueak, the parent
and child images are identical except for the result value of
the fork call.

> 
> If some objects which were originally present in the forking image get
> changed in the forked image, those changes are local to that image
> memory only, is that correct?  I assume this is the case since you
> mentioned "copy-on-write".

Yes that's right. Both images start with separate and identical object
memories (except for the #forkSqueak result value). Actual operating
system memory is consumed only when one or the other image changes
its object memory. In actual practice, the memory usage seems to be
quite modest, but this will of course vary depending on what the
images are doing. Calling a blocking FFI method in a child image
would be an example of something that would be very memory efficient.
Doing a full garbage collect in one of the images, probably not so
good.

> If the forked image returns one of the objects that was originally
> present in the forking image (modified or not), I assume it is
> returned as a "very deep copy" of the original, since you said it uses
> ReferenceStream to instantiate the result answer.

Yes, it can be any object that can be passed through a reference
stream, so it might for example be a matrix of Float values if you
were doing some sort of numeric work.

> 
> Finally, you mentioned it "should" work on Mac, but is there any
> reason it shouldn't work on Windows?

The OSProcess support for Windows is not sufficient to support this.
But OS X is a Unix, and the Mac VMs use the Unix OSProcessPlugin, so
I expect (hope?) that it will work there. If somebody tries it, please
let me know.

Dave

> 
> Thanks for this handy utility for easily distributing processing!
> 
>   Chris
> 
> 
> On Sun, Nov 13, 2011 at 9:21 AM, David T. Lewis <lewis at mail.msen.com> wrote:
> > RemoteTask provides partitioning of processing tasks at the level of
> > cooperating OS processes. For problems that can be partitioned into
> > independent tasks running under the supervision of a Squeak image, such
> > that each task performs a significant amount of processing and returns
> > a moderately sized result, RemoteTask can provide substantial improvements
> > in processing run time versus the equivalent serial processing when
> > running on multi-core hardware.
> >
> > A task is scheduled in a working image with
> >
> > ??RemoteTask do: taskBlock whenComplete: aOneArgumentBlock
> >
> > where taskBlock is the task to be scheduled in a forked Squeak image,
> > and aOneArgumentBlock handles the result object when the remote task
> > makes data available. Result data is returned through a ReferenceStream
> > on the stdout pipe from the remote Squeak image. The forked Squeak image
> > is headless and is quite memory efficient due to Unix copy-on-write
> > for forked processes.
> >
> > RemoteTask may also be useful for evaluating a method that otherwise would
> > block the VM, such as an FFI call to a long-running external function.
> >
> > RemoteTask is part of the latest CommandShell package on SqueakSource
> > (CommandShell 4.5.0), and requires OSProcess as well as a Unix or Mac VM
> > with OSProcessPlugin (it is helpful if the VM also has AioPlugin for process
> > completion notification, although polling will be used if this is not present).
> > I have tested only on Linux with standard VM and Cog, although I am hopeful
> > that this will work on Mac also (confirmation would be appreciated).
> >
> > Following is an example of three processing tasks assigned to three Squeak
> > worker images with results returned to the supervisory image on task
> > completion.
> >
> > threeParallelTasks
> > ?? "Find all primes in a range of large integers. Divide the problem into
> > ?? three tasks running the three child images, and return the results to
> > ?? the supervisory image. Answer a tasks array and a results array, where
> > ?? the results array will be populated on completion of the tasks."
> >
> > ?? "RemoteTask threeParallelTasks"
> >
> > ?? | p1 p2 p3 results task1 task2 task3 |
> > ?? results := Array new: 3.
> > ?? task1 := [(100000000000000000000000000000
> > ?? ?? ?? ?? ?? ?? ?? to: 100000000000000000000000019999)
> > ?? ?? ?? ?? ?? ??select: [:f | f isPrime] thenCollect: [:s | s asString]].
> > ?? task2 := [(100000000000000000000000020000
> > ?? ?? ?? ?? ?? ?? ?? to: 100000000000000000000000039999)
> > ?? ?? ?? ?? ?? ??select: [:f | f isPrime] thenCollect: [:s | s asString]].
> > ?? task3 := [(100000000000000000000000040000
> > ?? ?? ?? ?? ?? ?? ?? to: 100000000000000000000000059999)
> > ?? ?? ?? ?? ?? ??select: [:f | f isPrime] thenCollect: [:s | s asString]].
> > ?? "n.b. Assign task to a variable to prevent RemoteTask from being finalized"
> > ?? p1 := RemoteTask do: task1 whenComplete: [:result | results at: 1 put: result].
> > ?? p2 := RemoteTask do: task2 whenComplete: [:result | results at: 2 put: result].
> > ?? p3 := RemoteTask do: task3 whenComplete: [:result | results at: 3 put: result].
> > ?? ^ { #tasks -> { p1 . p2 . p3 } . #results -> results }
> >
> >
> > Dave
> >
> >
> >