[squeak-dev] Thoughts on Xtreams

Chris Cunnington brasspen at gmail.com
Mon Oct 5 16:04:45 UTC 2015



On 2015-10-03 1:45 AM, Colin Putney wrote:
>
>
> On Fri, Oct 2, 2015 at 8:06 AM, Chris Cunnington <brasspen at gmail.com 
> <mailto:brasspen at gmail.com>> wrote:
>
>     I do not have a mental model for what Xtreams is doing yet. How do
>     other people conceptualize using Xtreams in contrast to the
>     existing Streams implementation?
>
>
> I think the elegance of Xtreams boils down to one thing: composition 
> is better than inheritance.
>
> If you look at the Squeak stream hierarchy, it's a nightmare. Among 
> the highlights:
>
>   * There's a whole subhierarchy dedicated to compression. It
>     successfully shares code for different types of compression but
>     since DeflateStream inherits from PositionalStream, it can only
>     write compressed data into memory. If you want to write compressed
>     data to a file, you'll have to write to a compression stream, get
>     its contents and write that to a file stream.
>   * On the flip side, there's CrLfFileStream, which converts line
>     endings when reading and writing to a file. Except wait, it's
>     obsolete now and CrLfFileStream new actually returns an instance
>     of MultiByteFileStream. This class has an annoying name, because
>     it's camel-cased on syllable boundaries as well as word
>     boundaries. Ugh. Worse, it combines line-end conversion with
>     encoding conversion, but only when it text mode. Well, most of the
>     time, when in text mode. You gotta be careful about those few
>     methods that manipulate the file position in terms of bytes,
>     because that can leave it in the middle of a multibyte character
>     and then nothing works right. And if you want to do any of this
>     conversion on data in memory, you're outta luck because
>     MultiByteFileStream only works on data in files.
>   * Luckily MultiByteBinaryOrTextStream is here to save the day.
>     (Again with the capitals on syllable breaks.) It *does* work on
>     data in memory. It has a whole separate implementation of the
>     encoding and line-ending conversion code, plus no-nops
>     implementations of the file-related stuff in MultiByteFileStream
>     so the two are polymorphic. So convenient.
>   * There's also ReadWriteStream, which subclasses WriteStream, and
>     re-implements all of ReadStream's functionality.
>   * There's also SocketStream, for convenience in doing network IO. Oh
>     wait, it's not part of the Stream hierarchy at all. Never mind.
>
> There's more (lots more), but let's not get sidetracked. The point is 
> that there is just no way to have a sane inheritance hierarchy for a 
> whole bunch of orthogonal concerns:
>
>   * the underlying data storage - memory, file, socket or something
>     more exotic
>   * data transformation - encoding, compression, buffering, chunking etc
>   * reading vs writing
>
> Where Squeak streams try to do everything in one object, and combine 
> different options via inheritance, Xtreams splits a stream into a 
> pipeline of objects that each provide a separate bit of functionality. 
> It's so much more flexible.
I think I'm starting to see Xtreams like this.

A block can be used as a filter for iterating over the elements of a 
stream. If you have different filter blocks, then you can wrap each of  
them in class and have a library of different filters. If you compose 
these classes, then you have a pipeline of blocks that does a variety of 
things at once.

This has a functional feel because you cannot stop and examine things, 
because a filter has no state. It's just passing things on. Once it's 
composed, you need to wait until the end to see what happened. As such, 
it can handle infinite strings, as suggested in the SICP.

> In the specific case of Altitude, Xtreams provide two main advantages:
>
> First, the framework can build a custom pipeline of streams based on 
> message headers. To handle a request, we just examine the headers, 
> build the appropriate sequence of transformation streams and hand that 
> off to the application for reading.
> When the response is ready, we again look at the headers, build a 
> stream that performs all the transformations that the app has 
> indicated it wants, and let the app write into it. We can use any of 
> the features of HTTP, while still providing a simple and consistent 
> interface to the app

request := ALAuthenticationRequest new.
aRequest readEntityWith: [:in| ALJsonParser parse: in for: request ]

This was doing my head in. I expected all the stream contents to be 
rallied in the aRequest location, the whole stream would be there, so I 
could see it. But the Relay/Transforms phase is not over at that point. 
There is no state. It's not until the data is all in the 
ALAuthenticationRequest that I can examine it. That block is like 
another Xtreams filter. Stateless. All it does is sort things as they go 
by. And that's why I was confused. Foiled expectations.

> Second, it really helps with perceived performance. Browsers have 
> incredibly optimized strategies for processing data as it arrives from 
> the network. They're really good at rendering a page incrementally, 
> adding content and refining the appearance as more data arrives. 
> Xtreams allows Altitude to take advantage of that by doing the same 
> thing on the server side. As the app is rendering content, we push it 
> through the transformation streams in small chunks, and get it on the 
> network as quickly as possible. (ALStreamingExample is a good demo of 
> this.)
I think the word here is chunking. Since HTTP/1.1 came out I imagine 
clients have been optimizing for this. Servers? Not so much. I imagine 
that servers have never worked very hard to send data with the fine 
level of granularity equivalent to what clients can receive.
But I suppose this is really about buffer size and flushing frequency. I 
imagine Seaside could chunk its responses, but instead saves every 
response into one large buffer and then flushes once.
With Altitude I can put 'self halt' in the middle of a page and watch 
half a a page render. The buffer size is set to 1K and flushes when the 
buffer is full, which doesn't take long.

Chris

>
> Of course, there's a lot of other little details to like about 
> Xtreams, I think composition is what really makes it shine.
>
> Colin
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151005/b6a409b1/attachment.htm


More information about the Squeak-dev mailing list