concurrent processes (was: Re: [ANN] Magma 1.0r40)

[Chris Muller]

Hi Sig!

> I found code, which uses MagmaOidManager in two different processes. I
> placed a check instead of Mutex in it, which throws error, if some
> other process instead of one, which was first who accessed it tries to
> deal with oids.
>
> After running update, i found, that the only process, which accessing
> oid manager except one, which running update is running by magma
> itself:
>
> See fork in MaObjectRepository>>flushCacheSoon.
>
> After changing this method to:
>
> flushCacheSoon
>         self flushCritical:
>                         [ self commitCritical:
>                                 [ self flushCache.
>                                 applyProcess _ nil ] ]
>
> there's no other processes who attempt to use oid manager from except
> one, which runs update.
>
> So, the conclusion is obvious: flush causing problems.

Thanks for digging in and talking with code, this really helps us have
a productive discussion.

Thankfully, I don't see evidence of any Magma bug in what you've
submitted.  Your #checkProcess does not account for the fact that
flushCacheSoon is forked repeatedly. Your ifFalse: condition is being
satisfied against a dead process that already finished out. To serve
its intended purpose, you should check that the 'process' inst-var is
not terminated, as in:

checkProcess
	process ifNil: [ process := Processor activeProcess ] ifNotNil: [
		(process ~= Processor activeProcess
		and: [ process isTerminated not ] ]) ifTrue: [ self error: 'Multiple
processes using id manager' ].
	]

I'd be surprised if you hit your error with this change.

> Cris, please check the #flushCritical: code. I think it should not run
> inside a commit or parallel to commits if we want it to run it in
> standalone process.
> Otherwise magma disregards own rule which says that following is not allowed:
> session commit: [
>   [ some code ] fork
> ]

#flushCache does not run parallel with commits.  See in
#flushCacheSoon, the #flushCache is layered inside commitCritical:.

> Also, i'm glad to note, that with changes in #flushCacheSoon, Florian
> able to run own update without errors regardless using my dictionaries
> or using stock classes.

You probably fixed it by something else..  flushCache does not even
use MagmaOidManager, it just uses a standard OidManager.

> But timing are different:
> - out of the box magma: 18 minutes
> - using my dictionaries: 8 minutes.
>
> quite impressive for real world application isn't? :)

Indeed, I'd like the next release (r41) should be a performance
release that includes improved Dictionary's if possible..

> Cris, in attachment i placed the MagmaOidManager code, which checks
> the process. Just for case, if you want to see it yourself.

To make the test effective w.r.t. the flushCache code, you should move
your 'process' variable, and #checkProcess up to MaOidManager and call
it from that level rather than the Magma-specific subclass.

Regards,
  Chris