Am 22.03.2006 um 13:57 schrieb goran@krampe.se:

Hi!

Bert Freudenberg bert@impara.de wrote:

...

Hi folks,

Göran assured me that Magma was the Best Thing Since Sliced Bread so

Yup. :)

Regarding issues with loading - I am not using 1.1

Am I?

- Bert -

Everyone is running 1.0. Please see this later post:...

Shoot I could've sworn I copied this to the list, now I can't find it. Pasted here...

----- Forwarded Message ---- From: Chris Muller afunkyobject@yahoo.com To: Petr Sent: Sunday, March 12, 2006 4:41:40 PM Subject: Re: unable to get Magma working

There has been enough confusion about this that I have gone ahead and reverted MagmaClientLoader, MagmaServerLoader and MagmaTesterLoader back to the 1.0 versions. I'm pretty sure no one is using 1.1 yet so I will post here when the second attempt at security has been posted to SqueakMap.

Petr, I think most people still using 1.0 because a review of 1.1 has determined some adjustments are needed for security. I am working on these adjustments. In the meantime, you should use 1.0.

Try loading again and you will have the 1.0 versions which do not prompt for password.

- Chris

PS - the reason it prompts for the password twice; once when the #shutDown code is run, it prompts you to save your keyring in case you wish to change your password. The #startUp code has to reload your keyring, therefore you must enter your password.

Saving the image runs #shutDown followed by #startUp, which is why you were prompted twice. It has generated sufficient confusion that I am looking for a better way to handle this; after all most of the time don't want to change the password.

Am 22.03.2006 um 19:58 schrieb Chris Muller:

I hope you have fun and success with Magma.

Well, I converted my server in less than a day - not bad at all. It's some nice peace of software, and the documentation on the wiki was sufficient to get me started, thanks :)

Now I need to tune it. The initial import took well more than an hour, but now loading the root object is quite fast, as well as a commit. Much faster than before, which was the main point of switching.

However, there is one central report touching almost the whole model, which I had to interrupt after a quarter of an hour. It was taking way too long., I guess Magma was bringing in objects and throwing them out over and over again. I might easily be wrong though.

What strategy is in place to stub out objects? Actually I don't need any weak caching, I just need the commit. The whole model fits easily into main memory. Can I disable that? If not, how could I find out what's going on?

- Bert -

Am 23.03.2006 um 04:36 schrieb Chris Muller:

Hey Bert, what OS are you using? (my gut tells me Mac).. You may be blazing new trails here, I have not heard any experience reports from Mac users. Anyone?

Indeed, I develop on Mac OS X, deploying on Linux, though. You think I should just try it on the production machine?

...

However, there is one central report touching almost the whole model, which I had to interrupt after a quarter of an hour. It was taking way too long., I guess Magma was bringing in objects and throwing them out over and over again. I might easily be wrong though.

This should not happen, the ReadStrategy is a default minimumDepth of 1. It awards a +1 for Collections, so unless your model has any Collections of Collections it should stop reading.

It's basically a dictionary of (SSProject) objects each possessing a dictionary of (SSVersion) objects, whose attributes are also needed for that listing. This is around 6K objects times the attributes, which are complex themselves ... it really is touching like 50K or even 100K objects, and not only once but iterating several times. Which is absolutely no problem if everything is in the image.

I attach a message tally below. Bringing in one SSProject with 7 SSVersions took 27 seconds. Extrapolating this to the 6K SSVersions in total would take ... 2 days. Bummer.

If this is still a problem can you share a bit more about this?

If it is, in fact, a ReadStrategy issue, interruptting it after a long time, and then asking your session for the #cachedObjectCountByClass, may help provide a clue to the issue.

cachedObjectCountByClass is hanging (or taking ages), too. There seems to be a problem with the WeakValueDictionaries.

In the attached tally, Magma spends 97.3% (!) of the time in removeGarbageCollectedObjectEntries.

So you suspect it has to do with the Mac VM's fixed weak refs? But that would mean the Magma caching strategy is buggy as it relies on a buggy VM. Also, as soon as a full GC is required we'ld run into the same problem again, no?

And, I ran the MessageTally below with an older, 3.7 VM, which I think does not have the weak ref fix. So this might be something totally different, but I need to ask John if this VM has the fix or not.

...

What strategy is in place to stub out objects? Actually I don't need any weak caching, I just need the commit. The whole model fits easily into main memory. Can I disable that? If not, how could I find out what's going on?

WriterBarrier can make a significant difference, but does increase complexity..

How would I use that?

Otherwise, having the entire model in memory may slow down commits because a "comparison" is performed on every in-memory object to its original buffer for each commit.

Commits are rare, so this might be an option. How much memory overhead is introduced by the buffer comparison? Do you keep the original buffer around, then just serialize the current object to a new buffer and do a byte compare? Or is it rather like an MD5 hash of the original buffer that is compared to a hash of the object, which could be computed on the fly without actually allocating a new buffer?

You can use MagmaSession>>#stubOut: to chop off large branches of the model (leaving a proxy stub) to keep this operation lean.

Hmm, I'm not sure if it would be safe to stub out anything - the root is shared by all Seaside sessions, so one might modify the stuff another session is stubbing out afterwards. Or does the stub out method check if the subtree was modified? I guess not ...

- Bert -

MessageTally for bringing in 7 (of more than 6K) SSVersion objects.

- 25283 tallies, 26177 msec.

**Tree** 100.0% {26177ms} SSProject>>versions 58.5% {15314ms} MagmaMutatingProxy(MaMutatingProxy)

...

doesNotUnderstand:

|58.5% {15314ms} MagmaMutatingProxy(MaMutatingProxy)

...

mutateAndSend:

| 58.5% {15314ms} MagmaMutatingProxy>>mutant | 58.5% {15314ms} MagmaSession>>realObjectFor: | 58.5% {15314ms} MagmaSession>>materializeObject: | 58.5% {15314ms} MaObjectSerializer>>materializeGraph:do: | 58.5% {15314ms} MaObjectSerializer>>materializeGraphDo: | 58.4% {15287ms} MaObjectSerializer>>newSkeletonFor:fromProxy: | 58.1% {15209ms} MaObjectSerializer>>oidOf:is: | 58.1% {15209ms} MagmaOidManager>>oidOf:is: | 58.1% {15209ms} MagmaOidManager(MaOidManager)

...

oidOf:is:

| 58.1% {15209ms} MagmaOidManager>>removeGarbageCollectedObjectEntries | 58.1% {15209ms} MagmaOidManager (MaOidManager)>>removeGarbageCollectedObjectEntries | 58.1% {15209ms} WeakValueDictionary (Collection)>>reject: | 58.1% {15209ms} WeakValueDictionary (Dictionary)>>select: | 57.0% {14921ms} WeakValueDictionary>>add: [56.1% {14685ms} WeakValueDictionary>>at:put: [ 41.8% {10942ms} WeakValueDictionary(Set)>>findElementOrNil: [ |41.4% {10837ms} WeakValueDictionary(Dictionary)>>scanFor: [ 10.4% {2722ms} WeakValueDictionary(Set)>>atNewIndex:put: [ |10.0% {2618ms} WeakValueDictionary(Set)>>fullCheck [ | 9.2% {2408ms} WeakValueDictionary(Set)>>grow [ | 7.5% {1963ms} WeakValueDictionary(Dictionary)>>noCheckAdd: [ | 7.0% {1832ms} WeakValueDictionary(Set)>>findElementOrNil: [ | 6.5% {1702ms} WeakValueDictionary(Dictionary)>>scanFor: [ 3.5% {916ms} WeakValueAssociation class>>key:value: [ 2.4% {628ms} WeakValueAssociation>>key:value: [ 2.1% {550ms} primitives 41.5% {10863ms} SSVersion>>propertyAt:ifAbsent: 41.5% {10863ms} Dictionary>>DoItwith: 41.5% {10863ms} Dictionary>>DoItwith: 41.5% {10863ms} Dictionary>>DoIt 41.5% {10863ms} MagmaSession>>realObjectFor: 41.5% {10863ms} MagmaSession>>materializeObject: 41.5% {10863ms} MaObjectSerializer>>materializeGraph:do: 41.5% {10863ms} MaObjectSerializer>>materializeGraphDo: 41.4% {10837ms} MaObjectSerializer>>newSkeletonFor:fromProxy: 39.2% {10261ms} MaObjectSerializer>>oidOf:is: |39.2% {10261ms} MagmaOidManager>>oidOf:is: | 39.2% {10261ms} MagmaOidManager (MaOidManager)>>oidOf:is: | 39.2% {10261ms} MagmaOidManager>>removeGarbageCollectedObjectEntries | 39.2% {10261ms} MagmaOidManager (MaOidManager)>>removeGarbageCollectedObjectEntries | 39.2% {10261ms} WeakValueDictionary (Collection)>>reject: | 39.2% {10261ms} WeakValueDictionary (Dictionary)>>select: [38.5% {10078ms} WeakValueDictionary>>add: [ 38.0% {9947ms} WeakValueDictionary>>at:put: [ 29.0% {7591ms} WeakValueDictionary(Set)>>findElementOrNil: [ |28.8% {7539ms} WeakValueDictionary(Dictionary)>>scanFor: [ 7.0% {1832ms} WeakValueDictionary(Set)>>atNewIndex:put: [ 6.7% {1754ms} WeakValueDictionary(Set)>>fullCheck [ 6.1% {1597ms} WeakValueDictionary(Set)>>grow [ 4.6% {1204ms} WeakValueDictionary(Dictionary)>>noCheckAdd: [ 4.3% {1126ms} WeakValueDictionary(Set)>>findElementOrNil: [ 4.0% {1047ms} WeakValueDictionary(Dictionary)>>scanFor: 2.2% {576ms} MaVariableObjectBuffer (MaObjectBuffer)>>oid **Leaves** 80.7% {21125ms} WeakValueDictionary(Dictionary)>>scanFor: 3.2% {838ms} Array(SequenceableCollection)>>do: 2.7% {707ms} WeakValueAssociation>>key:value: 2.5% {654ms} MaVariableObjectBuffer(MaObjectBuffer)>>oid

**Memory** old -240,892 bytes young -170,488 bytes used -411,380 bytes free +479,116 bytes

**GCs** full 3 totalling 819ms (3.0% uptime), avg 273.0ms incr 907 totalling 873ms (3.0% uptime), avg 1.0ms tenures 167 (avg 5 GCs/tenure) root table 0 overflows

Am 25.03.2006 um 21:12 schrieb Chris Muller:

Now that the performance regression is restored, let me try responding properly.

...

In the attached tally, Magma spends 97.3% (!) of the time in removeGarbageCollectedObjectEntries.

Magma maintains a two-way map, object->oid (a.k.a. "oids") and oid-

...

object (a.k.a. "objects"). It currently handles this with a

WeakIdentityKeyDictionary and a MaWeakValueDictionary, respectively. WeakKey dictionaries automatically remove their (key-

...

value) entries when objects are collected/finalized, but WeakValue

dictionaries entries remain, and simply reference nil (i.e., oid-

...

nil).

Shouldn't then objects and oids usually have the same size?

So, as the client moves to explore a other parts of a huge domain model, the WeakKey dictionary is fine but the WeakValue never shrinks unless Magma cleans it up manually. This is what removeGarbageCollectedObjectEntries does. It simply rebuilds the entire MaWeakValueDictionary by enumerating all and only keeping the non-nil values.

The best strategy for when to rebuild is difficult to generalize. Time-based could cause unnecessary rebuilds or not rebuild soon enough, so I made based on conditions. Whenever the 'objects' size is twice that of the 'oids' size, the next oid assignment will take a detour rebuild the "objects" collection (oid->object), removing all the nil entries.

The problem for me is that in one of the three MagmaOidManager instances, objects size is 17516 and oids size is 6587 even after a full GC. Thus, removeGarbageCollectedObjectEntries is called on each- and-every access, explaining the huge time this takes.

What I did is adding a quick test to removeGarbageCollectedObjectEntries to check if there are indeed garbage collected entries (there are non in my initial load). This reduced the time spent in there from 97% to 12%, which is bearable. Raising the check threshold to "objects size > (oids size * 3)" made it go away alltogether - see attached MessageTally for bringing in 1K SSVersions.

I put my modified Magma packages at http://source.impara.de/ss.html

I also fixed the use of instVarNames instead of allInstVarNames in the MaReadStrategy, you could not name an inst var of a superclass otherwise.

- Bert -

89444 tallies, 90098 msec.

**Tree** 100.0% {90098ms} SSProject>>versions 96.0% {86494ms} SSVersion>>propertyAt:ifAbsent: |96.0% {86494ms} MagmaMutatingProxy(MaMutatingProxy)

...

doesNotUnderstand:

| 96.0% {86494ms} MagmaMutatingProxy(MaMutatingProxy)

...

mutateAndSend:

| 96.0% {86494ms} MagmaMutatingProxy>>mutant | 96.0% {86494ms} MagmaSession>>realObjectFor: | 93.1% {83881ms} MagmaSession>>materializeObject: | |92.9% {83701ms} MaObjectSerializer>>materializeGraph:do: | | 92.9% {83701ms} MaObjectSerializer>>materializeGraphDo: | | 91.3% {82259ms} MaObjectSerializer>>newSkeletonFor:fromProxy: | | 90.6% {81629ms} MaVariableObjectBuffer (MaObjectBuffer)>>oid | | 90.6% {81629ms} primitives | 2.9% {2613ms} MagmaSession>>read: | 2.9% {2613ms} MagmaSession>>submit: | 2.8% {2523ms} MaLocalServerLink>>submit: | 2.8% {2523ms} MaLocalRequestServer (MaRequestServer)>>processRequest: | 2.8% {2523ms} MagmaRepositoryController>>value: | 2.8% {2523ms} MagmaRepositoryController>>processRequest: | 2.8% {2523ms} MaReadRequest>>process | 2.8% {2523ms} MaObjectRepository>>read:for:using: | 2.8% {2523ms} MaObjectRepository>>byteArrayAt:using: | 2.7% {2433ms} MaObjectRepository>>graphBufferAt:using: | 2.5% {2252ms} MaVariableObjectBuffer (MaObjectBuffer)>>appendGraphUsing:into:...:minDepth:with:filer: 4.0% {3604ms} MagmaMutatingProxy(MaMutatingProxy)>>doesNotUnderstand: 4.0% {3604ms} MagmaMutatingProxy(MaMutatingProxy)>>mutateAndSend: 4.0% {3604ms} MagmaMutatingProxy>>mutant 4.0% {3604ms} MagmaSession>>realObjectFor: 2.4% {2162ms} MagmaSession>>materializeObject: 2.4% {2162ms} MaObjectSerializer>>materializeGraph:do: 2.4% {2162ms} MaObjectSerializer>>materializeGraphDo: **Leaves** 90.7% {81719ms} MaVariableObjectBuffer(MaObjectBuffer)>>oid 3.1% {2793ms} Dictionary>>scanFor:

**Memory** old +2,303,788 bytes young +113,468 bytes used +2,417,256 bytes free -2,417,256 bytes

**GCs** full 0 totalling 0ms (0.0% uptime) incr 370 totalling 1,624ms (2.0% uptime), avg 4.0ms tenures 32 (avg 11 GCs/tenure) root table 0 overflows

Am 28.03.2006 um 08:20 schrieb Chris Muller:

...

I also fixed the use of instVarNames instead of allInstVarNames in the MaReadStrategy

Another good improvement, thanks!

Hmm, doesn't work though, I get a walkback in inheritDeeperDepthsUsing:.

My strategy was this:

(MaReadStrategy minimumDepth: 1) forVariableNamed: 'timestamp' onAny: SSVersion readToDepth: 99999; forVariableNamed: 'properties' onAny: SSVersion readToDepth: 99999; forVariableNamed: 'downloadStatistics' onAny: SSVersion readToDepth: 99999;

Inheritance is this:

Object #() SSModel #('timestamp') SSVersion #('author' 'versionInfo' 'dependencies' 'properties' 'downloadStatistics')

My workaround was to add this to my read stategy:

forVariableNamed: 'timestamp' onAny: SSModel readToDepth: 0;

And wow does using the right strategy make a difference! I'm down from 700 to 100 seconds initial load time.

Debug log attached.

- Bert -

Am 25.03.2006 um 21:12 schrieb Chris Muller:

Equally important, how much to keep in memory by hard-referencing or not. Hard referencing the root for example will cause all parts of the database explored to remain in memory, growing and growing endlessly. This may be tempting to do so that objects don't have to be "reread" but it also causes the dictionary's to get very big and slow. Were it not for MaWeakValueDictionary replacing WeakValueDictionary, the "objects" dictionary becomes unusable after just a couple hundred-thousand entries or so..

My "objects" are a plain WeakValueDictionary ... do I need to switch something on?

Management of the size of the in-memory domain model is also helped by #stubOut:. Magma does not call this itself, it is a tool for the developers discretion. It involves a becomeForward: so the best thing to do is, for example, stub the single Collection of 6000 objects, not each of the 6000. I don't think multiple Seaside sessions referencing an object you stubOut: should be a problem although may be good idea to guard it with a Mutex.

What if another session modifies something while a commit is in progress? Could this lead to problems? I guarded the commitAndBegin in a mutex, though.

Having said all this, though, your "straight port" to Magma, from the all-in-memory approach may still be workable if you can endure the initial "load time" of the model into memory (90% of which is probably building those darn dictionary's). Just be sure to KEEP it all in memory after its gone through all the trouble.. :) Commits will probably slow down so if they get too unbearable you could always turn on WriteBarrier.

Would using WriteBarrier be as simple as "myMagmaSession allowWriteBarrier: true"? What drawbacks are there?

- Bert -

Hi all!

Chris Muller chris@funkyobjects.org wrote:

When each Seaside session has its own copy of the model, multiple processes are not an issue, and Magma sees them as independent clients so the concurrency detection works too. It is the only approach with potential to scale to multi-machines / processes as described on the Magma Seaside page.

• Chris

I have been toying with having a single readonly session shared by multiple Seaside sessions (Processes) and then have a smaller pool of writeable sessions which I grab one from when I want to make modifications (commits).

In my scenario this could actually work out pretty simple because I use a Command pattern anyway (all of my modifications are done using messages to the top object which in turn instantiates "Commands" that are applied), and most users will tend to mostly read and not write, or at leat write pretty seldomly.

That means I could get away with one large (after a while it would get large unless I stub things out) readonly model + say 10 partial models (one for each writeable session in the pool).

So 1+10 models for serving say 150 concurrent sessions seems much better than 150 models. :)

regards, Göran