- Traversing a Reader 'hogs' memory on the server
This is our biggest problem. If I have a script which iterates over a reader, doing something with the tree reachable each object and then stubbing it out at the bottom of the loop, the server image's memory footprint just keeps growing. The client image's memory is stable.
e.g.
| reader | reader := (session root things read: #name). 1 to: reader size do: [ :i | | thing | thing := reader at: i. "do other things with thing here..." "materialize a lot of objects but no transactions" session stubOut: thing. session finalizeOids. Smalltalk garbageCollect. ].
I did find a memory "leak" that occurs for commits, I'll explain. We must now re-serialize commit-packages to support disaster-recovery, I was not "finalizingOids" of the serializer that takes care of this work. The attached change-set corrects this. (Note, in this case, you'll see I was actually able to use #resetOidManager, which is quicker for clean-up than #finalizeOids, but you not usable for a MagmaSessions serializer..).
I was not able to reproduce the case above only enumerating a reader to cause the server to leak memory.. Let me share my test scripts and, if it is still happening for you, perhaps you can modify the scripts such that they recreate the problem and then send them back to me..
** Script 1 - Create a huge MagmaSet of UUID's, indexed on #asString36.
| path s set | path _ 'c:\temp\bigger'. MagmaRepositoryController create: path root: (MagmaSet equivalenceIndex: ((MaSearchStringIndexDefinition attribute: #asString36) keySize: 192 ; yourself)). s _ MagmaSession openLocal: path. s connectAs: 'cmm'. s begin. set _ s root. 1 to: 1000000 do: [ : n | set add: UUID new. n \ 100 = 0 ifTrue: [ s commitAndBegin ; finalizeOids ] ]. s commit. s
NOTE, in the interests of time, I broke this after it had added 493000+ UUID's to run the next scripts..
** Script 2 - Start the server **
path _ 'c:\temp\bigger'. MagmaServerConsole new open: path ; processOn: 1010
** Script 3 - Enumerate (separate image) **
| path s set reader | path _ 'c:\temp\bigger'. s _ MagmaSession hostAddress: #(127 0 0 1) asByteArray port: 1010. s connectAs: 'chris'. set _ s root. reader := (s root read: #asString36). 1 to: reader size do: [ :i | | thing | thing := reader at: i. "do other things with thing here..." "materialize a lot of objects but no transactions" i\1000=0 ifTrue: [ Transcript show: '.' ]. ]. s
I let this run but the server hovered at 30MB constant for several minutes before I interrupted it.
My guess is you probably had the commit memory-leak affecting you. The attached change-set seems to fix it. I hope it fixes your memory-leak issue..
- Chris