Yes I applied his fix, but - DataStream saves a Bag naively (not as
item-count pairs). Saving 600K tokens this way is very wasteful.
Specializing the save could remove a 10 factor. Making the token saving
incremental (as the saving the index is) would make it near
instantanous.

About Celeste changes, I think it would be a very, very good idea to
delay all changes until we have an established removal script on SM, and
a Celeste base package on SM, which we can maintain there. Otherwise,
chaos will ensue.

Daniel

goran.hultgren at bluefish.se wrote:
> Daniel Vainsencher <danielv at netvision.net.il> wrote:
> [SNIP]
> > PS - if you want to make a lot of changes to Celeste, then it might be a
> > good idea to talk about them, since Lex, I think, will want to coevolve
> > his FilteringCeleste stuff with it, and I want to package up and improve
> > TimOlsons Bayesian Spam Filter (wonderful stuff, but saving it is too
> > slow when you use it for more than a couple of months. I want to speed
> > it by a factor of 10, or make it constant time). I also want to slowly
> 
> You did apply his fix right? He used Bag>>size inside a loop. Bad idea.
> I haven't profiled it yet but without that fix it got deadly slow.
> The hacks to get it to work with FilteringCeleste was easy.
> Hmmm, I need to do a list of things I would like to change/fix in
> Celeste...
> 
> regards, Göran