You sent this around a while back, but I tend to get behind on email, so
I'm responding anyway :)

Celeste really does take a long time to read/write its internal database
to/from disk.  It's okay on a very fast computer, but even there, it's
too slow for hooking it into mailto: link handling.  (Imagine clicking
on such a link by accident!)

There are two general approaches to making things faster.  First, you
can back off and have a wimpier database.  This is what most mail
programs do; the user must keep things sorted into relatively small
folders, and as long as they do this, the program will run pretty fast. 
Celeste has a much spiffier database, which lets you put messages into
multiple categories and which handles very large folders pretty well (to
the limits of PluggableListMorph's), but it pays a price for this.

The second approach would be to implement the database more quickly. 
There's no particular need for Celeste to load the *entire* index into
memory when it starts, so why not just have Celeste load things on
demand?  This sounds like a trivial database problem, but consider two
things:

	1. The records are very small
	2. The records have a natural clustering (categories) which should be
taken advantage of

In fact, being headstrong, I once started implementing a basic
BTree-based database for Celeste.  It took so long to read in the
couple-hundred records for my 'new' folder, that it wasn't really any
faster than just loading the entire index sequentially!  I could
actually hear the hard drive spattering all around, much like when
Netscape is started.


To get back to the specific problems you fixed, removing extra
updateTOC's sounds great, but I for one would like to keep around the
safety net that auto-saving the index provides.


Lex



Michael Rutenberg <mdr at scn.org> wrote:
> Change Set:		CelesteSpeedups
>Date:			3 January 2000
>Author:			Mike Rutenberg (mdr at scn.org)
> 
> These changes are important in speeding up Celeste mailboxes containing
> several thousand messages or when running on slower computers.
> 
> (1) Removes a redundant (and very costly) self changed: message from
> updateTOC.  When displaying a new/modified large category (e.g. the 8000
> Squeak messages of last year) it cuts the (Morphic) update time almost
> in half
> from 14 seconds.
>
> (2) Change to eliminate saving the category and index file to disk after
> retrieving new messages or sending messages.  The message contents
> continue to be immediately flushed to the message file when they are
> retrieved from the server or typed to be sent.  This change makes
> sending and retrieving messages significantly (!!!!) faster.
> 
> Discussion of possible dangers of (2):
> I believe that this introduces no new dangers, while significantly
> improving the performance of Celeste.  There are a number of ways the
> index might differ between Squeak and disk, such as when you move a
> message between categories.   In the case of Squeak crashing before the
> index does get saved, choosing 'compact' from the categories menu will
> rebuild the index file based on the contents of the message file.  In
> the worst case, it is possible to construct a scenario in which a Squeak
> crash would result in outgoing messages being sent twice.

> Goal for the future:
> Make saving/restoring the Celeste index file to disk so fast that it can
> be done with any change