On 2-Dec-09, at 8:26 AM, Nicolas Cellier wrote:

> Xtream is not functional yet, it is just a three evenings shot.

At the rate you're going, you'll have caught up to the functionality  
in Filesystem in no time. ;-)

> Especially pipelines are quite tricky with a forked process... I got
> to rest a bit and think.
> This kind of implementation natively has good parallelism properties,
> unfortunately this won't exploit multi-core/processors any time soon
> in Smalltalk...

I'm guessing you mean running each stage of the stream in a separate  
Smalltalk Process, using the Pipes and Filters pattern?
Stephen Pair did some neat stuff with that a few years ago. It's  
indeed tricky. I wonder if it's worth it in this case, though, exactly  
because Smalltalk doesn't exploit multiprocessors. Flow of control  
inside a stream might be complicated without parallelism, but it's  
probably easier to debug.

> A few month ago, I implemented a simple Wrapper-like scheme, but was
> not satisfied with end of stream handling. Both EndOfStream exception
> capture and atEnd tests are expensive when processing elements 1 by 1.
> Maybe I'll have to turn to such a more simple scheme though.

I don't understand the issue with EndOfStream exceptions. Throwing and  
catching an exception is expensive, yes, but that should happen only  
once, right? Unless you're setting up exception handler inside a loop,  
the expense of a single exception shouldn't be a problem.

> Definitely, we should exchange code/ideas.

Agreed. We may find that doing parallel development with lots of cross- 
pollination is the best way to explore the design space.

Colin