[Seaside] Re: Yes, it works - and fast too! (was [Q] File Upload/Download Server, Comanche or Swazoo)

[Göran Krampe]

Hi again!

Janko Mivšek wrote:
> Hi all,
> 
> Göran's work encouraged me to run benchmarks more carefully and
> repeatably to minimize impact of OS file system performance and as you
> can see, Kom and Swazoo are actually close in upload performance, about
> 5-6MB/s.
> 
> Big apology to Squeak community for that original 1.5MB/s figure, which
> obviously came out from only one test, instead of repeatable to avoid
> other impacts. In new tests I  didn't change Swazoo code, I just made an
> additional upload demo on Aida.
> 
> Here are results in seconds of repeatable test for two files, 572MB and
> 102MB ones:
> 
> 572MB file (600.037.00 bytes)
> 
>   Squeak:
>     Kom/Seaside   92  98 117 (avg 5.9MB/s)
>     Swazoo/Aida  108 111 107 (avg 5.6MB/s)
>   VW:
>     Swazoo/Aida   67  68  66 (avg 9MB/s)
> 
> 
> 102MB file (106.491.000 bytes)
> 
>   Squeak:
>     Kom/Seaside   18  35  25  32 (avg 3.9MB/s)
>     Swazoo/Aida   16  16  17  16 (avg 6.6MB/s)
>   VW:
>     Swazoo/Aida    5   5   6   5 (avg 20MB/s)
> 
> As you can see for larger files we are very close while for smaller file
> Swazoo is still faster. Also you can see that Kom results vary a lot
> while Swazoo ones are stable. It seems that this is due to bigger amount
> of garbage generation by Kom (this can also be seen from memory
> consumption variations) while Swazoo garbage is minimal.
> 
> In any case we can conclude with quite certainty that on Squeak we
> reached the upload speed limit near 6MB/sec. To be faster we will need
> to optimize TCP socket and file VM modules.

Hmmm, ok, I did two more things:
- Refactored away one buffer copy. Now the Kom code copies straight from
the SocketStream inBuffer into the FileStream.
- Raised the buffer size from 1 Mb to 10 (or 20, 30, it does improve
numbers with a large buffer).

This gave the following numbers on my box for some files:

Time: 11.613 Size: 81601394 MB/s: 6.70121016453518
Time: 0.064 Size: 642560 MB/s: 9.57489013671875
Time: 12.024 Size: 81601394 MB/s: 6.4721518330628
Time: 0.111 Size: 828738 MB/s: 7.12023554621516
Time: 130.331 Size: 731654144 MB/s: 5.353751337939554
Time: 126.742 Size: 731654144 MB/s: 5.50535549087911

I also profiled this a bit and now there is very little gc going on and
I can't really easily see how to improve this much further. Small files
seem to generally be faster, but as you see the 81Mb file lands
regularly between 6.5 and 7 MB/s. Peaks around 10Mb for small files.

Latest version attached, this was tested with Seaside 2.8.

regards, Göran
-------------- next part --------------
A non-text attachment was scrubbed...
Name: StreamedUploadZeroCopy.1.cs.gz
Type: application/x-gzip
Size: 3271 bytes
Desc: not available
Url : http://lists.squeakfoundation.org/pipermail/seaside/attachments/20080803/4b701dc5/StreamedUploadZeroCopy.1.cs.bin