[Seaside] Uploading large files
mkobetic at cincom.com
Tue Aug 14 14:28:42 UTC 2007
If you are using VW, the new Seaside-Opentalk (note, we renamed the packages from SeasideForOpentalk to Seaside-Opentalk-*) will automatically stream any part of incoming request that has fileName specified in its header directly into a file. The default behavior is to save any such part into a configured directory under the provided name. If a file with that name exists already we will attach _v1 to the name for the new part, then _v2, etc. This behavior can be configured. Note that at this point we don't attempt to manage the contents of the directory in any way, we just create new files as they come in.
The way this works is that the component that parses incoming requests (HttpBuildHandler) has a bodyStreamBlock, which gets invoked before each simple body is read from the socket and is expected to create a stream to write the body into. The block receives a MimeEntity corresponding to the part being processed (with headers) and the builder (useful to access any configuration parameters) as arguments. It is up to the block to decide if an internal or external stream is created. To change the default block, you have to modify builderBlock on your SeasideTransportConfiguration. This is another block because there's a new builder instance created for each incoming request. The builderBlock is responsible for creating new builder instances, so you can set a new bodyStreamBlock on the builder instance directly in the builderBlock.
For any part of an incoming request that has a fileName we will create a WAFileStream (instead of WAFile). It is a trivial extension of WAFile which maintains its contents as a stream instead of a collection (byte array or string). WAFile would force us to read the contents of the files into memory when we create WARequest, obviously defeating the purpose of the optimizations described above. Note however that using #contents on WAFileStream will trigger reading of the whole file (currently every time you send it #contents). I'm not sure if and how often that happens once the WARequest is handed off to Seaside. I think it would be preferable to modify WAFile to use a stream internally and modify all such places to use the stream instead of #contents. Of course you should also avoid using #contents in your fileUploadWithCallback: callbacks as well, use WAFileStream>>stream instead.
If you have any comments about the solution I described above, please let me know.
Chad Nantais wrote:
> What's the smartest way to handle the uploading of large files (>50mb)
> through Seaside and onto the filesystem without grossly inflating my
> image's memory consumption in the process?
> Thanks in advance,
> Seaside mailing list
> Seaside at lists.squeakfoundation.org
More information about the seaside