Hey!
I got some interesting results from fiddling with various project zipping strategies on the XO.
Basically using only the unzipped project file, big projects load about 3 times faster than the doubly zipped alternative (where the project zip bundle is not compressed); which is the way project files are stored now.
Numbers are in seconds, horizontally we see the zipping methods and vertically three different projects. On a clean 3.0 image, loaded with our customizations and luke's speed patch.
1 2 3 4 5 tr 37 1.20 1.29 1.29 46 c10 30 1.15 1.23 1.23 40 shp 18 30 32 36 20
where tr = triangle identification c10 = adding to 10 is fun shp = counting sheep
1 = non-zipped stand-alone .pr file 2 = zipped .pr file in uncompressed archive 3 = non-zipped .pr file in uncompressed archive 4 = non-zipped .pr file in compressed archive 5 = zipped stand-alone .pr file
Testing the other 1.20m plus activities also shows a threefold speedup to generally just under 40 sec against the 2.3 image with luke's dictionary patch loaded.
For us at OLE the obvious choice is 1, cause we basically just go for raw speed, but I guess there's something to say for 5, because it does cut the size of the files in half. To me it seems worth it on the XO, but others might not be as performance inclined. Or?
In general, fascinatingly enough, doubly zipped projects seem to load faster than bundles with a zipped .pr file in them.
Attached the patches that made this work:
The patches I already sent. Useful irrelevant of the patches that follow them in this post: fileIn Fix: for unzipped .pr files the wrong kind of stream is created (i think).
AsUnzippedStream: don't use unzip routine with built in progress bar. Speedup of about 20% on the larger files. Note that this doesn't have anything to do with the project progress bar.
new ones (names are a bit misleading). Basically just throw in the ones whose effect you want. For a combination of bundle and uncompressed .pr file, make sure you've got the fileIn Fix loaded: zipOptimizations: don't zip the inner .pr file
furtherZipOptimizations: don't zip the bundle and don't create the other files in the bundle (html file, etc...).
compress project bundle: If filed in, the project bundle is compressed.
Hope any of this is of use to some.
/Ties
Ties Stuij wrote:
Hey!
I got some interesting results from fiddling with various project zipping strategies on the XO.
Basically using only the unzipped project file, big projects load about 3 times faster than the doubly zipped alternative (where the project zip bundle is not compressed); which is the way project files are stored now.
Numbers are in seconds, horizontally we see the zipping methods and vertically three different projects. On a clean 3.0 image, loaded with our customizations and luke's speed patch.
1 2 3 4 5
tr 37 1.20 1.29 1.29 46 c10 30 1.15 1.23 1.23 40 shp 18 30 32 36 20
where tr = triangle identification c10 = adding to 10 is fun shp = counting sheep
1 = non-zipped stand-alone .pr file 2 = zipped .pr file in uncompressed archive 3 = non-zipped .pr file in uncompressed archive 4 = non-zipped .pr file in compressed archive 5 = zipped stand-alone .pr file
Testing the other 1.20m plus activities also shows a threefold speedup to generally just under 40 sec against the 2.3 image with luke's dictionary patch loaded.
For us at OLE the obvious choice is 1, cause we basically just go for raw speed, but I guess there's something to say for 5, because it does cut the size of the files in half. To me it seems worth it on the XO, but others might not be as performance inclined. Or?
In general, fascinatingly enough, doubly zipped projects seem to load faster than bundles with a zipped .pr file in them.
Attached the patches that made this work:
The patches I already sent. Useful irrelevant of the patches that follow them in this post: fileIn Fix: for unzipped .pr files the wrong kind of stream is created (i think).
AsUnzippedStream: don't use unzip routine with built in progress bar. Speedup of about 20% on the larger files. Note that this doesn't have anything to do with the project progress bar.
new ones (names are a bit misleading). Basically just throw in the ones whose effect you want. For a combination of bundle and uncompressed .pr file, make sure you've got the fileIn Fix loaded: zipOptimizations: don't zip the inner .pr file
furtherZipOptimizations: don't zip the bundle and don't create the other files in the bundle (html file, etc...).
compress project bundle: If filed in, the project bundle is compressed.
Hope any of this is of use to some.
/Ties _______________________________________________ Etoys mailing list Etoys@lists.laptop.org http://lists.laptop.org/listinfo/etoys
I think if we could keep form instances compressed in the image, we could have reasonably small projects without compression.
Karl
On 10.04.2008, at 06:05, Ties Stuij wrote:
Hey!
I got some interesting results from fiddling with various project zipping strategies on the XO.
Basically using only the unzipped project file, big projects load about 3 times faster than the doubly zipped alternative (where the project zip bundle is not compressed); which is the way project files are stored now.
Cool!
Numbers are in seconds, horizontally we see the zipping methods and vertically three different projects. On a clean 3.0 image, loaded with our customizations and luke's speed patch.
pr pr.gz.zip0 pr.zip0 pr.zip pr.gz
tr 37 1:20 1:29 1:29 46 c10 30 1:15 1:23 1:23 40 shp 18 30 32 36 20
where tr, c10, shp are different projects
pr = non-gzipped stand-alone .pr file pr.gz.zip0 = gzipped .pr file in uncompressed zip archive pr.zip0 = non-gzipped .pr file in uncompressed zip archive pr.zip = non-gzipped .pr file in compressed zip archive pr.gz = gzipped stand-alone .pr file
I just reformatted your table for my own understanding. So apparently avoiding the zip-archive part is paramount for performance. We should figure out what's actually taking so long there ...
- Bert -
Bert Freudenberg wrote:
On 10.04.2008, at 06:05, Ties Stuij wrote:
Hey!
I got some interesting results from fiddling with various project zipping strategies on the XO.
Basically using only the unzipped project file, big projects load about 3 times faster than the doubly zipped alternative (where the project zip bundle is not compressed); which is the way project files are stored now.
Cool!
Numbers are in seconds, horizontally we see the zipping methods and vertically three different projects. On a clean 3.0 image, loaded with our customizations and luke's speed patch.
pr pr.gz.zip0 pr.zip0 pr.zip pr.gz
tr 37 1:20 1:29 1:29 46 c10 30 1:15 1:23 1:23 40 shp 18 30 32 36 20
where tr, c10, shp are different projects
pr = non-gzipped stand-alone .pr file pr.gz.zip0 = gzipped .pr file in uncompressed zip archive pr.zip0 = non-gzipped .pr file in uncompressed zip archive pr.zip = non-gzipped .pr file in compressed zip archive pr.gz = gzipped stand-alone .pr file
I just reformatted your table for my own understanding. So apparently avoiding the zip-archive part is paramount for performance. We should figure out what's actually taking so long there ...
- Bert -
It would be interesting to see how much time it takes to save projects with the different compressions, too.
Karl
Numbers are in seconds, horizontally we see the zipping methods and vertically three different projects. On a clean 3.0 image, loaded with our customizations and luke's speed patch.
pr pr.gz.zip0 pr.zip0 pr.zip pr.gz
tr 37 1:20 1:29 1:29 46 c10 30 1:15 1:23 1:23 40 shp 18 30 32 36 20
where tr, c10, shp are different projects
pr = non-gzipped stand-alone .pr file pr.gz.zip0 = gzipped .pr file in uncompressed zip archive pr.zip0 = non-gzipped .pr file in uncompressed zip archive pr.zip = non-gzipped .pr file in compressed zip archive pr.gz = gzipped stand-alone .pr file
I just reformatted your table for my own understanding. So apparently avoiding the zip-archive part is paramount for performance. We should figure out what's actually taking so long there ...
Yes. a stand-alone .pr file would lose a few other files in the archive, right? I would hope that we can find a way to make "pr.zip0" or "pr.zip" perform almost as fast as "pr"...
-- Yoshiki
etoys-dev@lists.squeakfoundation.org