2009/12/2 Colin Putney <cputney at wiresong.ca>:
>
> On 2-Dec-09, at 8:26 AM, Nicolas Cellier wrote:
>
>> Xtream is not functional yet, it is just a three evenings shot.
>
> At the rate you're going, you'll have caught up to the functionality in
> Filesystem in no time. ;-)
>

Hmm, I need to rest too...

>> Especially pipelines are quite tricky with a forked process... I got
>> to rest a bit and think.
>> This kind of implementation natively has good parallelism properties,
>> unfortunately this won't exploit multi-core/processors any time soon
>> in Smalltalk...
>
> I'm guessing you mean running each stage of the stream in a separate
> Smalltalk Process, using the Pipes and Filters pattern?
> Stephen Pair did some neat stuff with that a few years ago. It's indeed
> tricky. I wonder if it's worth it in this case, though, exactly because
> Smalltalk doesn't exploit multiprocessors. Flow of control inside a stream
> might be complicated without parallelism, but it's probably easier to debug.
>

Yes

>> A few month ago, I implemented a simple Wrapper-like scheme, but was
>> not satisfied with end of stream handling. Both EndOfStream exception
>> capture and atEnd tests are expensive when processing elements 1 by 1.
>> Maybe I'll have to turn to such a more simple scheme though.
>
> I don't understand the issue with EndOfStream exceptions. Throwing and
> catching an exception is expensive, yes, but that should happen only once,
> right? Unless you're setting up exception handler inside a loop, the expense
> of a single exception shouldn't be a problem.
>

yes, my naive implementation of stream wrappers did have to handle the
exception inside the loop.
I don't remember why...
More other, I first used Notification, and it was dangerous because
uncaught Notification can hit an upper handler.
The case when the number of elements change thru a Wrapper can be
tricky (think of a select: operation)
Maybe I should have a refreshed look at it now.

Now I would handle it with an endOfStreamAction anyway...

somewhere in initialization:
    endOfStreamMark := Object new.
    source endOfStreamAction: nil->endOfStreamMark.

and then:
next
   | anElement |
   ^(anElement := source next) = endOfStreamMark
        ifTrue: [endOfStreamAction value]
        ifFalse: [anElement]

my own endOfStreamAction has been set upper in the chain presumably.

Nicolas

>> Definitely, we should exchange code/ideas.
>
> Agreed. We may find that doing parallel development with lots of
> cross-pollination is the best way to explore the design space.
>
> Colin
>
>

One day, we may converge :)

Nicolas