[ENH] Celeste speedup for many messages
Lex Spoon
lex at cc.gatech.edu
Tue Jan 18 15:35:07 UTC 2000
You sent this around a while back, but I tend to get behind on email, so
I'm responding anyway :)
Celeste really does take a long time to read/write its internal database
to/from disk. It's okay on a very fast computer, but even there, it's
too slow for hooking it into mailto: link handling. (Imagine clicking
on such a link by accident!)
There are two general approaches to making things faster. First, you
can back off and have a wimpier database. This is what most mail
programs do; the user must keep things sorted into relatively small
folders, and as long as they do this, the program will run pretty fast.
Celeste has a much spiffier database, which lets you put messages into
multiple categories and which handles very large folders pretty well (to
the limits of PluggableListMorph's), but it pays a price for this.
The second approach would be to implement the database more quickly.
There's no particular need for Celeste to load the *entire* index into
memory when it starts, so why not just have Celeste load things on
demand? This sounds like a trivial database problem, but consider two
things:
1. The records are very small
2. The records have a natural clustering (categories) which should be
taken advantage of
In fact, being headstrong, I once started implementing a basic
BTree-based database for Celeste. It took so long to read in the
couple-hundred records for my 'new' folder, that it wasn't really any
faster than just loading the entire index sequentially! I could
actually hear the hard drive spattering all around, much like when
Netscape is started.
To get back to the specific problems you fixed, removing extra
updateTOC's sounds great, but I for one would like to keep around the
safety net that auto-saving the index provides.
Lex
Michael Rutenberg <mdr at scn.org> wrote:
> Change Set: CelesteSpeedups
>Date: 3 January 2000
>Author: Mike Rutenberg (mdr at scn.org)
>
> These changes are important in speeding up Celeste mailboxes containing
> several thousand messages or when running on slower computers.
>
> (1) Removes a redundant (and very costly) self changed: message from
> updateTOC. When displaying a new/modified large category (e.g. the 8000
> Squeak messages of last year) it cuts the (Morphic) update time almost
> in half
> from 14 seconds.
>
> (2) Change to eliminate saving the category and index file to disk after
> retrieving new messages or sending messages. The message contents
> continue to be immediately flushed to the message file when they are
> retrieved from the server or typed to be sent. This change makes
> sending and retrieving messages significantly (!!!!) faster.
>
> Discussion of possible dangers of (2):
> I believe that this introduces no new dangers, while significantly
> improving the performance of Celeste. There are a number of ways the
> index might differ between Squeak and disk, such as when you move a
> message between categories. In the case of Squeak crashing before the
> index does get saved, choosing 'compact' from the categories menu will
> rebuild the index file based on the contents of the message file. In
> the worst case, it is possible to construct a scenario in which a Squeak
> crash would result in outgoing messages being sent twice.
> Goal for the future:
> Make saving/restoring the Celeste index file to disk so fast that it can
> be done with any change
More information about the Squeak-dev
mailing list
|