[Seaside] large files / 2 - WAFile

Michal miso.list at auf.net
Mon Apr 12 00:34:23 CEST 2004


> I finally got around to looking at this code, and it's not quite what I 
> expected - it looks like it still reads the entire file into memory 
> from the socket, but then immediately dumps it to disk if it's too 
> large.  This is still going to be problematic if multiple people are 
> uploading large files at once, isn't it?  Wouldn't it be better to 
> stream directly off the socket onto disk?

I went for the minimal change that would work (ie. stop the image from
latching on to huge files long after they are uploaded). The change I
attached solves that, hopefully in a minimal way. But:

On the one hand, you're right of course that directly streaming to a
file would be best. At first sight, that should be a fairly simple
change: create a method just like #saveToStream: (eg #saveToFile:) and
call that from #processMultipartFields:. (A quick look at Colin's
changeset makes it look like it is not there yet, as it uses my
#fileContents which will suck the entire file into a string. It is
also more complicated than it needs to be, it seems to me).

On the other hand, I'm not implementing that, as I'm going in the
radically opposite direction: I have discovered that reading the
entire http request upfront, together with an optimisastion for
reading large chunks from sockets, will speed up things by a factor of
several hundred for uploads of big files (!). This is in fact
something that Bolot had already implemented in #rawPostFields in the
original Comanche, but only for 'application/x-www-form-urlencoded'
requests. It is relatively simple to extend that to
'multipart/form-data' - which I have done.

That solution still leaves hanging the case where several uploads of
big files take place simultaneously, but given the dynamic growth of
images, today's RAM sizes and the tenuous likelihood of such a
situation, I'm not worried by that. (The original issue was that the
image *kept* the whole file in memory as part of Seaside's holding on
continuations - a different issue altogether)

Michal


More information about the Seaside mailing list