[ANN] Magma 1.0r40

[Igor Stasenko]

On 26/09/2007, Chris Muller <asqueaker at gmail.com> wrote:
> > I'm placed a line in removeGarbageCollectedObjectEntries
> > Transcript show: 'Oids count: ' , self oidCount asString; cr.
> > And during running Florian's update it shows (between other lines):
> > Oids count: 29923
> > Oids count: 45449
> > Oids count: 49919
> > Oids count: 66399
> > Oids count: 76492
> > So its growing and growing..
> > And only way to get rid of dead keys, is to call MaOidManager>>finalizeOids.
> > But i can't find any places, leading to call of this method. Is this
> > designed to be managed by user directly, using
> > MagmaSession>>finalizeOids ?
>
> Yes, as documented in the performance tuning page
>
>     http://wiki.squeak.org/squeak/2985
>
> at this time finalizeOids is intended to be called by the user,
>
> > I think it should call finalizeValues to dicts in
> > removeGarbageCollectedObjectEntries, so both dicts with object>id and
> > id>object associations will be cleaned and rehashed.
>
> but that sounds like a good idea.  I know these two cleaning
> mechanisms were tacked on years apart from each other, so maybe that's
> why they ended up separate mechanisms.
>
> Of course, all this is just to accomodate less than ideal Weak
> Dictionary's included with Squeak.
> I'd much rather fix or replace the Dictionary's.  Florian, have you
> tried any alternative Weak Dictionary's available?  Be sure to check
> out Martin Loewis' and Sig's.  (I've posted the link to Martin's
> several times in the last months, it's on Mantis).
>
> > Or, maybe, since magma uses own version(s) of weakIdentityKey dicts,
> > implementation of #at:put: in weakIdentityKey dict can be changed to
> > silently reuse entries with key=nil. So there will be much less need
> > for rehashing.
>
> MaDictionary is a dead-simple hack around the 12-bit identity hash
> limitation.  Whenever standard WeakIdentityDictionary's get over 4K
> elements their performance suffers tremendously, so MaDictionary makes
> the degradation more linear by managing a collection of <4K sized
> Dictionary's.
>
> I really don't want to make the Ma Dictionary's more complicated.  I
> really would just like to replace them with something built from the
> ground up to scale to more than 4K elements.
>
> > it. But then i wonder, why with big pauses putted in #reject: (or not
> > using #reject: , as in new case) helped to avoid errors?
>
> I have never seen the errors myself.  Do they happen with the stock
> Magma MaDictionary's?  If so, how can I reproduce it?
>
> > Interesting, how well magma handles following code:
> > session commit: [
> >   [ big long huge loop ] fork.
> >   [ big long huge loop 2 ] fork.
>
> Do NOT do that.  However, you should be able to do this:
>
> [ big long huge loop with multiple commits ] fork.
> [ big long huge loop 2 with multiple commits ] fork.
>
> > Is it allows to spawn different processes in commit?
>
> No.

So, my suggestions was correct. Since we can end up with using
different classes/packages which interact with magma, and they, in
turn, can use third party packages, you never really know, if such
"malicious" code executing or not.

>
> > I now trying to run update with all methods in MagmaOidManager
> > enclosed by mutex.
>

Yesterday i sent changed MagmaOidManager with mutex to Florian, and he
reported that he was able to run his update (twice in a row) without
any errors.

Also, i tried to run update on image which Florian gave me month ago,
with mutexes and stumbled upon error which says, that something
changing object(s) while they being serialized. This is another sign,
that there is unknown entity doing something wrong with data , and
most probably runs in parallel. I can't say what is it, or why or
where it is.

Personally, i don't like an idea of having mutex in MagmaOidManager
because it slows the code which used very frequently. I'm just worry
that this may be the only way to keep things safe.
Of course its a question , what Florian does in his update process,
which leads to such errors, and what he can do to avoid that without
using mutexes.

> Thanks Sig, I hope we can find a lean, mean, fast Dictionary machine
> for the next Magma.
>
>  - Chris
>


-- 
Best regards,
Igor Stasenko AKA sig.