On Wed, Dec 02, 2009 at 05:26:53PM +0100, Nicolas Cellier wrote:
> 
> Xtream is not functional yet, it is just a three evenings shot.
> Especially pipelines are quite tricky with a forked process... I got
> to rest a bit and think.

Are you referring to pipelines in the sense of unix command pipelines?
This is indeed tricky, and requires for sure that streams on pipes
be set to non-blocking mode, otherwise you lock up the VM (*). If
you have the CommandShell package loaded, you may want to look at
the tests in category "testing-pipelines" in CommandShellTestCase
for examples.

In CommandShell, the necessary methods are just hacked into subclasses
of StandardFileStream. The basic approach is to set non-blocking
mode for pipe reads, then use AIO notification to signal data
available. The process that is reading data from the pipe waits on
a semaphore, then reads the available data. This prevents the VM
lockup problem, and allows lots of processes to concurrently read
from pipe streams.

I'm sure the StandardFileStream hack is not what you want, but it
might provide some ideas.

(*) Of course if you use threads in the VM, this would be different,
but that is another subject I think.

Dave