[Lex wants a full in memory index to make filters run fast]
Well, sometimes I do too :-)
Well, if we make the cache size a parameter, you can set it to 1e6, and
thus cache everything. The only down side is the two files will be
slightly bigger than having one unified file as we do now. Probably
still smaller than the current indices because of the efficient binary
encoding of numbers.

Anyway, this leaves us lots of room to play - and I expect to find that
setting it to 1e3/4 works fine for me.

Is this okay for you, Lex, guys?

About the cache entry format - I like it being self descriptive as you
propose. I'm pretty sure that making the file binary will get us most of
the speed improvement. Celeste already knows how to spit out/read in a
binary number - for the categories file. I'm not sure we need more code
to do that (or maybe I just didn't understand the part about reading
multiple numbers).

The important thing to make strings fast is not to use CrLfStream...
it's detection mechanism backfires horribly when you're switch between
binary and text modes. Just keep everything binary, and we're fine. 

BTW, we need one more field type - the date. It's can be written as a
number but requires proper wrapping in the image.

Daniel