Hi Chris, hi all, I'm testing my app with many objects by bulk loading it. When I do it for 1250 simple object transactions it works fine but when I start with the first of ten files with 42000 simple transactions by each I got this error so I "Proceed" and I got it again, and again. Finally a got Out of memory because Croquet.exe is about 500 mb of ram. All is working fine, process the file, creating all objects and adding to my application but the problem is when I do commit: to it.
With this information could you guess what i'm doing wrong?
Thanks in advance, FV
p.s.: in my test i would like bulk load 42000 customers with 10 simple transactions by each. MaObjectSerializationWarning: Serialization sizeWarningThreshold reached. Proceed to double it.
Debug http://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&1 Proceedhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&2 Full Stackhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&3 Stack Trace
1. thisContexthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&4 MaVariableObjectBuffer(MaObjectBuffer)>>ensureSpaceFor:using:selfhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&5a MaVariableObjectBuffer oid : 0 classId : 0 objectInstSize : -3 commitNumber : MaBufferPositio...etc...storageObjecthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&6a Dictionary(size 47600)bpMapClasshttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&7 MagmaBufferPositionMaptotalSizehttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&8 nilrequiredSpacehttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&9 nil 2. thisContexthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&10 MaObjectSerializer>>bufferFor:storageObject:startingAt:selfhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&11a MaObjectSerializer anObjecthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&12a Dictionary(size 47600)storageObjecthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&13a Dictionary(size 47600)anIntegerhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&14 9691714bufferhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&15 nil 3. thisContexthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&16 MaObjectSerializer>>append:selfhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&17a MaObjectSerializer anObjecthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&18a Dictionary(size 47600)storageObjecthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&19 nil 4. thisContexthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&20[] in MaObjectSerializer>>appendGraph:do:selfhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&21a MaObjectSerializer oneArgBlockhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&22an OrderedCollection(a MaIdentitySet(size 94307) a Dictionary(size 47600)) pathhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&23a MaIdentitySet(size 94307)parenthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&24 50581indexhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&25 nil 5. thisContexthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&26 MaIdentitySet(ProtoObject)>>maValueGraphNode:index:using:with:path:with: selfhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&27a MaIdentitySet(size 94307)anObjecthttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&28a Dictionary(size 47600)anIntegerhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&29 50581aObjectTraversalStrategyhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&30a MaObjectGraphTraversalStrategyaBlockhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&31[closure] in MaObjectSerializer>>appendGraph:do:pathhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&32an OrderedCollection(a MaIdentitySet(size 94307) a Dictionary(size 47600)) alreadyVisitedSethttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&33a MaIdentitySet(size 171342)stepIntohttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&34 nilobjhttp://localhost:9090/sic?_s=LJ8Qe-PiB3ZoG1q6&_k=4vp3vHPmR8w2JVGS&35 nil
Hi Facundo,
You may try smaller commits? A script like the following could work to commit every 100 additions to a MagmaCollection:
addToCollectionAt: aDirectoryName collection: aCollection
| myMagmaSession commitIntervalCount magmaCol | myMagmaSession := self sessionClass openLocal: aDirectoryName. myMagmaSession connectAs: self authorName. magmaCol := myMagmaSession root first. commitIntervalCount := 100. 1 to: aCollection size by: commitIntervalCount do: [: startIndex | | stopIndex | ( stopIndex := startIndex + commitIntervalCount - 1 ) <= aCollection size ifTrue: [ myMagmaSession begin. ( aCollection copyFrom: startIndex to: stopIndex ) do: [: obj | magmaCol add: obj ]. myMagmaSession commit ]. ]
You may pre-set the sizeWarningThreshold to a bigger value too.
Cheers,
2011/3/26 Facundo Vozzi facundov79@gmail.com:
Hi Chris, hi all,
I'm testing my app with many objects by bulk loading it. When I do it for 1250 simple object transactions it works fine but when I start with the first of ten files with 42000 simple transactions by each I got this error so I "Proceed" and I got it again, and again. Finally a got Out of memory because Croquet.exe is about 500 mb of ram. All is working fine, process the file, creating all objects and adding to my application but the problem is when I do commit: to it. With this information could you guess what i'm doing wrong? Thanks in advance, FV p.s.: in my test i would like bulk load 42000 customers with 10 simple transactions by each.
MaObjectSerializationWarning: Serialization sizeWarningThreshold reached. Proceed to double it.
Debug Proceed Full Stack
Stack Trace
thisContextMaVariableObjectBuffer(MaObjectBuffer)>>ensureSpaceFor:using: selfa MaVariableObjectBuffer oid : 0 classId : 0 objectInstSize : -3 commitNumber : MaBufferPositio...etc...storageObjecta Dictionary(size 47600)bpMapClass MagmaBufferPositionMaptotalSizenilrequiredSpace nil thisContextMaObjectSerializer>>bufferFor:storageObject:startingAt:self a MaObjectSerializer anObjecta Dictionary(size 47600)storageObject a Dictionary(size 47600)anInteger9691714buffer nil thisContextMaObjectSerializer>>append:self a MaObjectSerializer anObjecta Dictionary(size 47600)storageObject nil thisContext[] in MaObjectSerializer>>appendGraph:do:self a MaObjectSerializer oneArgBlockan OrderedCollection(a MaIdentitySet(size 94307) a Dictionary(size 47600)) patha MaIdentitySet(size 94307)parent 50581indexnil thisContext MaIdentitySet(ProtoObject)>>maValueGraphNode:index:using:with:path:with:selfa MaIdentitySet(size 94307) anObjecta Dictionary(size 47600)anInteger 50581aObjectTraversalStrategya MaObjectGraphTraversalStrategyaBlock [closure] in MaObjectSerializer>>appendGraph:do:pathan OrderedCollection(a MaIdentitySet(size 94307) a Dictionary(size 47600)) alreadyVisitedSeta MaIdentitySet(size 171342)stepInto nilobjnil
Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma
magma@lists.squeakfoundation.org
-
Facundo Vozzi -
Hernán Morales Durand