Magma on 3.7 report
Bert Freudenberg
bert at impara.de
Thu Mar 23 16:31:33 UTC 2006
Am 23.03.2006 um 04:36 schrieb Chris Muller:
> Hey Bert, what OS are you using? (my gut tells me Mac).. You may
> be blazing new trails here, I have not heard any experience reports
> from Mac users. Anyone?
Indeed, I develop on Mac OS X, deploying on Linux, though. You think
I should just try it on the production machine?
>> However, there is one central report touching almost the whole model,
>> which I had to interrupt after a quarter of an hour. It was taking
>> way too long., I guess Magma was bringing in objects and throwing
>> them out over and over again. I might easily be wrong though.
>
> This should not happen, the ReadStrategy is a default minimumDepth
> of 1. It awards a +1 for Collections, so unless your model has any
> Collections of Collections it should stop reading.
It's basically a dictionary of (SSProject) objects each possessing a
dictionary of (SSVersion) objects, whose attributes are also needed
for that listing. This is around 6K objects times the attributes,
which are complex themselves ... it really is touching like 50K or
even 100K objects, and not only once but iterating several times.
Which is absolutely no problem if everything is in the image.
I attach a message tally below. Bringing in one SSProject with 7
SSVersions took 27 seconds. Extrapolating this to the 6K SSVersions
in total would take ... 2 days. Bummer.
> If this is still a problem can you share a bit more about this?
>
> If it is, in fact, a ReadStrategy issue, interruptting it after a
> long time, and then asking your session for the
> #cachedObjectCountByClass, may help provide a clue to the issue.
cachedObjectCountByClass is hanging (or taking ages), too. There
seems to be a problem with the WeakValueDictionaries.
In the attached tally, Magma spends 97.3% (!) of the time in
removeGarbageCollectedObjectEntries.
So you suspect it has to do with the Mac VM's fixed weak refs? But
that would mean the Magma caching strategy is buggy as it relies on a
buggy VM. Also, as soon as a full GC is required we'ld run into the
same problem again, no?
And, I ran the MessageTally below with an older, 3.7 VM, which I
think does not have the weak ref fix. So this might be something
totally different, but I need to ask John if this VM has the fix or not.
>> What strategy is in place to stub out objects? Actually I don't need
>> any weak caching, I just need the commit. The whole model fits easily
>> into main memory. Can I disable that? If not, how could I find out
>> what's going on?
>
> WriterBarrier can make a significant difference, but does increase
> complexity..
How would I use that?
> Otherwise, having the entire model in memory may slow down commits
> because a "comparison" is performed on every in-memory object to
> its original buffer for each commit.
Commits are rare, so this might be an option. How much memory
overhead is introduced by the buffer comparison? Do you keep the
original buffer around, then just serialize the current object to a
new buffer and do a byte compare? Or is it rather like an MD5 hash of
the original buffer that is compared to a hash of the object, which
could be computed on the fly without actually allocating a new buffer?
> You can use MagmaSession>>#stubOut: to chop off large branches of
> the model (leaving a proxy stub) to keep this operation lean.
Hmm, I'm not sure if it would be safe to stub out anything - the root
is shared by all Seaside sessions, so one might modify the stuff
another session is stubbing out afterwards. Or does the stub out
method check if the subtree was modified? I guess not ...
- Bert -
MessageTally for bringing in 7 (of more than 6K) SSVersion objects.
- 25283 tallies, 26177 msec.
**Tree**
100.0% {26177ms} SSProject>>versions
58.5% {15314ms} MagmaMutatingProxy(MaMutatingProxy)
>>doesNotUnderstand:
|58.5% {15314ms} MagmaMutatingProxy(MaMutatingProxy)
>>mutateAndSend:
| 58.5% {15314ms} MagmaMutatingProxy>>mutant
| 58.5% {15314ms} MagmaSession>>realObjectFor:
| 58.5% {15314ms} MagmaSession>>materializeObject:
| 58.5% {15314ms} MaObjectSerializer>>materializeGraph:do:
| 58.5% {15314ms} MaObjectSerializer>>materializeGraphDo:
| 58.4% {15287ms}
MaObjectSerializer>>newSkeletonFor:fromProxy:
| 58.1% {15209ms} MaObjectSerializer>>oidOf:is:
| 58.1% {15209ms} MagmaOidManager>>oidOf:is:
| 58.1% {15209ms} MagmaOidManager(MaOidManager)
>>oidOf:is:
| 58.1% {15209ms}
MagmaOidManager>>removeGarbageCollectedObjectEntries
| 58.1% {15209ms} MagmaOidManager
(MaOidManager)>>removeGarbageCollectedObjectEntries
| 58.1% {15209ms} WeakValueDictionary
(Collection)>>reject:
| 58.1% {15209ms} WeakValueDictionary
(Dictionary)>>select:
| 57.0% {14921ms}
WeakValueDictionary>>add:
[56.1% {14685ms} WeakValueDictionary>>at:put:
[ 41.8% {10942ms} WeakValueDictionary(Set)>>findElementOrNil:
[ |41.4% {10837ms} WeakValueDictionary(Dictionary)>>scanFor:
[ 10.4% {2722ms} WeakValueDictionary(Set)>>atNewIndex:put:
[ |10.0% {2618ms} WeakValueDictionary(Set)>>fullCheck
[ | 9.2% {2408ms} WeakValueDictionary(Set)>>grow
[ | 7.5% {1963ms} WeakValueDictionary(Dictionary)>>noCheckAdd:
[ | 7.0% {1832ms} WeakValueDictionary(Set)>>findElementOrNil:
[ | 6.5% {1702ms} WeakValueDictionary(Dictionary)>>scanFor:
[ 3.5% {916ms} WeakValueAssociation class>>key:value:
[ 2.4% {628ms} WeakValueAssociation>>key:value:
[ 2.1% {550ms} primitives
41.5% {10863ms} SSVersion>>propertyAt:ifAbsent:
41.5% {10863ms} Dictionary>>DoItwith:
41.5% {10863ms} Dictionary>>DoItwith:
41.5% {10863ms} Dictionary>>DoIt
41.5% {10863ms} MagmaSession>>realObjectFor:
41.5% {10863ms} MagmaSession>>materializeObject:
41.5% {10863ms} MaObjectSerializer>>materializeGraph:do:
41.5% {10863ms} MaObjectSerializer>>materializeGraphDo:
41.4% {10837ms}
MaObjectSerializer>>newSkeletonFor:fromProxy:
39.2% {10261ms} MaObjectSerializer>>oidOf:is:
|39.2% {10261ms} MagmaOidManager>>oidOf:is:
| 39.2% {10261ms} MagmaOidManager
(MaOidManager)>>oidOf:is:
| 39.2% {10261ms}
MagmaOidManager>>removeGarbageCollectedObjectEntries
| 39.2% {10261ms} MagmaOidManager
(MaOidManager)>>removeGarbageCollectedObjectEntries
| 39.2% {10261ms} WeakValueDictionary
(Collection)>>reject:
| 39.2% {10261ms} WeakValueDictionary
(Dictionary)>>select:
[38.5% {10078ms} WeakValueDictionary>>add:
[ 38.0% {9947ms} WeakValueDictionary>>at:put:
[ 29.0% {7591ms} WeakValueDictionary(Set)>>findElementOrNil:
[ |28.8% {7539ms} WeakValueDictionary(Dictionary)>>scanFor:
[ 7.0% {1832ms} WeakValueDictionary(Set)>>atNewIndex:put:
[ 6.7% {1754ms} WeakValueDictionary(Set)>>fullCheck
[ 6.1% {1597ms} WeakValueDictionary(Set)>>grow
[ 4.6% {1204ms} WeakValueDictionary(Dictionary)>>noCheckAdd:
[ 4.3% {1126ms} WeakValueDictionary(Set)>>findElementOrNil:
[ 4.0% {1047ms} WeakValueDictionary(Dictionary)>>scanFor:
2.2% {576ms} MaVariableObjectBuffer
(MaObjectBuffer)>>oid
**Leaves**
80.7% {21125ms} WeakValueDictionary(Dictionary)>>scanFor:
3.2% {838ms} Array(SequenceableCollection)>>do:
2.7% {707ms} WeakValueAssociation>>key:value:
2.5% {654ms} MaVariableObjectBuffer(MaObjectBuffer)>>oid
**Memory**
old -240,892 bytes
young -170,488 bytes
used -411,380 bytes
free +479,116 bytes
**GCs**
full 3 totalling 819ms (3.0% uptime), avg 273.0ms
incr 907 totalling 873ms (3.0% uptime), avg 1.0ms
tenures 167 (avg 5 GCs/tenure)
root table 0 overflows
More information about the Magma
mailing list