Hi Everyone,
As Frank mentioned, I am indeed interested in starting to get quantified performance data up and being logged for Squeak on a regular basis. The motivation for this comes from visiting the site arewefastyet.com where Mozilla logs the Javascript performance of Firefox, and also from listening to a talk by Lars Bak, who worked on the Self VM and V8. He stated that one of the first things he does for any of his VM projects is to create a comprehensive set of performance benchmarks that is updated with every commit, because otherwise it is too easy for performance regressions to sneak in while development is occuring.
My hope is that by offering continuous feedback about the performance of Squeak, it will help provide a bit of a catalyst for anyone who is interested to go ahead and try to chip away at the performance of image or VM code. It might also help people understand Squeak's standing in the performance realm, relative to other languages.
I figure that I'll throw my goals out there, but I must caution that I have never worked with Jenkins before, so I will have some learning ramp up time before results start showing up:
- Get the performance plugin logging a simple dummy file written out by Squeak using the CI machinery.
- Use one of the tests that I have ported from the "Benchmarks Shoot-out game" to start getting a "real" number out and updating regularly, with a nice graph showing progress
- Get all of the benchmarks from the "Benchmarks Shoot-out game" ported and running in squeak to give an somewhat healthy spread of code.
- Get benchmarks for some other areas up and running. My inital thoughts would be to have some drawing benchmarks, so that I can quantify the performance of VPRI's Gezira graphics library vs Balloon/FormCanvas.
Stretch goals: - Get benchmarks for other critical parts of the system, maybe stream benchmarks so if we adopt Xtreams or something else we can quantify the impact of that, maybe some parsing benchmarks for XML, etc. - Get some benchmarks from Octane and other suites ported over - Have the server run the benchmarks for another language such as Python, or Ruby, or maybe Javascript so we can chart Squeak's progress against them - Make some contributions to Squeak to speed some of these benchmarks up!
Anyway, if anyone has some input or suggestions I would be grateful to receive them. If you think that my plan is sound, I would love to hear that as well.
Thanks for your time, Jeff!
On Wed, Jan 23, 2013 at 2:46 AM, Frank Shearar frank.shearar@gmail.comwrote:
Jeff's interested in answering "How fast is Squeak today?" letting us view trends on performance, not just on build times.
He's going to be taking the Performance plugin for a spin. This plugin renders JMeter format output.
frank