On Wed, Dec 2, 2009 at 1:49 PM, Colin Putney <cputney at wiresong.ca> wrote:

>
> On 2-Dec-09, at 8:26 AM, Nicolas Cellier wrote:
>
>  Especially pipelines are quite tricky with a forked process... I got
>> to rest a bit and think.
>> This kind of implementation natively has good parallelism properties,
>> unfortunately this won't exploit multi-core/processors any time soon
>> in Smalltalk...
>>
>
> I'm guessing you mean running each stage of the stream in a separate
> Smalltalk Process, using the Pipes and Filters pattern?
> Stephen Pair did some neat stuff with that a few years ago. It's indeed
> tricky. I wonder if it's worth it in this case, though, exactly because
> Smalltalk doesn't exploit multiprocessors. Flow of control inside a stream
> might be complicated without parallelism, but it's probably easier to debug.


Some time ago I implemented dynamic bindings, which gives you an ability to
get and set process local (or more accurately, stack local) variables.  I
had also implemented a SharedStream class that firmed up the semantics of
underflow vs end of stream as well as overflow (if you wanted to cap the
size of the buffer).  It was process safe (as the name implies) and had
blocking and non blocking protocols.  It could handle any kind of object and
was efficient for bytes and characters.

I later read a paper on a more expressive model for concurrency (can't
recall the author) that discussed conjunctive and disjunctive joints and I
created an implementation.  With the protocol I chose, you could do
something like:

   ([a doSomething] | [b doSomething]) value

That would evaluate both blocks concurrently and answer the first one to
complete (and terminate the laggard).  The stack is stitched together in
such a way that an exception signaled in either block would propagate out
and could be handled naturally with an on:do:...when in a handler, both of
the child processes would be suspended.  This was the disjunctive joint.
 Similarly, you could do:

   ([a doSomething] & [b doSomething]) value

This would evaluate both blocks concurrently, but not return an answer until
both completed (and it would return the answer of the last block).  It has
the same exception handling semantics.  This was the conjunctive joint.  You
could also chain as many blocks as you wanted to run them concurrently.

>From there it was a natural extension want to do something like:

   (someString >> [x grep: 'foo'] >> [x sort]) value

Where >> represented a pipe (in the unix sense) and used SharedStream and
the concurrency work to connect up the processes.  I used dynamic bindings
to define stdin and stdout for each of the processes.  With some contortions
I managed to reduce it to something like:

   (someString >> (Grep with: 'foo') >> Sort) value.

I imagine with implicit self sends (ala Self or Newspeak), it could be
reduced even further:

   (someString >> grep: 'foo' >> sort) value

- Stephen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20091203/3d8ab822/attachment.htm