On Thu, Nov 20, 2008 at 08:57:40AM -0700, Matthew Fulmer wrote:
On Wed, Nov 19, 2008 at 09:48:38PM -0800, Craig Latta wrote:
It's not the filesystem access that is slow. It is that the current source code storage format reads the file character by character, looking for the terminating !, and doing utf-8 conversion the whole way. 95% of the time in a MC snapshot is spent in testing the source file characters for the terminating ! character. I imagine you do much more in bulk rather than character by character. The format of the .sources file is horrible for access speed.
Oh, right! I forgot about that. :) Yeah, should be quite a bit
faster, I'm just answering String objects that are sitting in memory. The measurements will be interesting.
But again, I take it from your comment that you're actually
finding the speed of the traditional setup to be a problem? If so, then yeah, it'd make a good marketing point.
It's by far the biggest bottleneck in MC speed. Try viewing changes on the Morphic package. It takes about 3 minutes on a fast machine, and 99.6% of the time is source code lookup, last time I measured Morphic
I mention it because I've spent a lot of time optimizing MC for speed. Between MC 1.0 and 1.6, I've improved PackageInfo speed 8x and package loading speed 6x (and I have another unstable experimental patch that reduces loading from an quadratic time operation to linear time one). However, I havn't been able to touch package saving speed at all, because it would require changing the source code storage format, which is outside the scope of MC.