On 11 May 2010 19:21, Andreas Raab <andreas.raab at gmx.de> wrote:
> On 5/11/2010 7:52 AM, Igor Stasenko wrote:
>>
>> just one feature request: support streaming in WebClient,
>> i.e. apart from being able to fetch a response content in a single blob,
>> also provide an API, which would enable WebClient user to read content
>> step by step, by using stream protocol, i.e. #next, #next: etc.
>>
>> I see you having a #streamFrom:to:size:progress:
>> in WebMessage, but its a little bit too high level (requires an output
>> stream and includes a progress block)
>> and its end consumer is a #content message.
>>
>> It would be nice to have something like:
>>
>> response := WebClient httpGet: 'http://foo.bar'.
>>
>> content := response content. "read all at once".
>
> That works out of the box.
>
>> contentStream := response contentStream.  "read content using a stream"
>> [contentStream atEnd] whileFalse: [
>>   c := contentStream next.  ....
>> ].
>
> And that works out of the box, too :-) With a small modification. The
> convenience APIs on the class side close the client and prefetch the
> response. In other words, in order to stream you need to write, e.g,
>
>  client := WebClient new.
>  [resp := client httpGet:'http://www.squeak.org'.
>  length := resp contentLength.
>  stream := resp contentStream.
>  [length > 0] whileTrue:[
>    stream next: (length min: 100).
>  ]] ensure:[stream close].
>
good. Its quite similar to what i did in SCouchDB.

But i'd like to note, that content length is not mandatory field, and
your example
won't work for a content with undetermined length.

> A couple of things to keep in mind though: The above doesn't deal with the
> server returning an HTTP/1.0 response (no content-length) or other
> specialties like chunked-encoding content transfer. That's why the
> high-level functions are advantageous because they deal with all of that.
>

In SCouchDB I dealing with chunked-encoding content by wrapping the
original stream with chunked stream
which deals with this transparently, and so the end user don't have to
deal with it by itself.
Then, similarily, for utf-8 content, i also just wrapping a source
stream with own utf8-decoder stream,
and so, the content consumer don't have to care about it by itself,
all it needs to know to use stream protocol.

So, stream-wrappers is a powerful abstraction, alas, the current
Stream package lack of good support for 'wrapper stream' such kind of
abstraction. :(

> Cheers,
>  - Andreas
>
>

-- 
Best regards,
Igor Stasenko AKA sig.