[Seaside] support for streamed files with Zinc adaptor?

Philippe Marschall philippe.marschall at gmail.com
Wed Jun 20 05:41:16 UTC 2012


On Tue, Jun 19, 2012 at 10:06 PM, Sven Van Caekenberghe <sven at beta9.be> wrote:
> Hey Philippe,
>
> On 19 Jun 2012, at 20:15, Philippe Marschall wrote:
>
>> On Tue, Jun 19, 2012 at 4:12 PM, Sven Van Caekenberghe <sven at beta9.be> wrote:
>>> Hi Esteban,
>>>
>>> On 19 Jun 2012, at 15:31, Esteban Lorenzano wrote:
>>>
>>>> Hi Sven el all
>>>>
>>>> I'm using Zinc-Adaptor and I need to upload files. I remember Comanche adaptor had support for streamed files (they were derived to a temporary file instead load everything in the image).
>>>> While I could configure nginx or apache to do that, I would like to be able to do it  using plain Seaside too.
>>>> It is possible with Zinc-Adaptor?
>>>>
>>>> thanks,
>>>> Esteban_______________________________________________
>>>
>>> I am (still) confused as to what exactly is expected of a 'streaming' Seaside adaptor.
>>
>> There are two different issues, file upload streaming and response streaming.
>>
>> What I believe is wanted here is file upload streaming. The idea is
>> that instead of loading the whole contents of a file upload into
>> memory (and copying it several times along the way) you stream the
>> contents into a file on disk and just get a "handle" to the file.
>> Dealing with multipart mime makes this a bit tricky, you don't want to
>> do this for normal upload fields. So you need to hook into your mime
>> parser. It's basically the same as mod_upload does [2]. Looking for
>> references to WATemporaryFile in an up to date Seaside image [1]
>> should get you started. Note that the Comanche implementation relies
>> on the fact that the request is dispatched after the headers are
>> parsed before the body is fully parsed.
>
> Thanks for the excellent, to the point explanation.
> I browsed the code and found what I was looking for.
>
> This won't be easy as it depends on, first a streaming multipart parser (the one currently used is based on your AJP parser that works on a byte array, do you have another one ?),

No, as AJP works only behind a front end server you'd probably do this
is with a module there instead of in the Smalltalk image. So I don't
have a need for it.

> second in Zn currently only generic byte streams can be seen as streaming entities, not interpreted/parsed ones like multipart, I'll have to think about that.

If you encode the multipart boundary with the request encoding you
should be able to do the parsing on byte streams, in theory, I haven't
tested it.

> And then indeed, there is the interaction between the server and the adaptor. Hmm. a challenge indeed.

You don't have to do the decoding in the adaptor like Comanche. You
can do all the processing in the server and then pass a special entity
to the adaptor that knows about it.

>> The other thing is response streaming, sending data to the client
>> before the response is fully computed. An example for this is to send
>> the html header when you start rendering the body so that the user
>> agent can start downloading linked resources. If you don't know how
>> long the response is going to be (because it still being build) you'll
>> have to resort to chunking. See WAComboResponse >> #flush to get you
>> started.
>
> Very interesting as well.
> Is this working ? Using which adaptor ?

Comanche, sort of. WAComboResponse implements http chunking [1] and
writes directly to the socket stream. Once done it signals to Comanche
that the response has already been sent and it shouldn't do anything
(which is probably bad, it should probably close the connection if
requested). You don't have to chunking if:
 - the client doesn't want or support keep alive (eg. HTTP 1.0)
 - theres a connection: close header in the response, you won't
normally do this but it can be useful for EventSource implementations,
streaming iframes and so forth

But you don't have to use WAComboResponse, you can use a custom
subclass of WAResponse that understands #flush and does the right
thing.

 [1] http://en.wikipedia.org/wiki/Chunked_transfer_encoding

Cheers
Philippe


More information about the seaside mailing list