magma handle memory

[Chris Muller]

HI Juan,

I really need to add a "How to do a Bulk Load" page to the documentation, sorry.

In the meantime, you weren't clear what you were adding the 19K trees
of objects *TO*.  My guess is you are adding to a Smalltlak
collection; a Dictionary or OrderedCollection perhaps?  It's good you
are doing the stubbing and finalizieOids, but they won't help much if
the root collection is a Smalltalk collection because the entire
persistent graph is in memory then (except for the stubbed, maybe).

You need to add them to a MagmaCollection instead, so that transparent
paging of objects in and out of memory can occur.

To verify what kinds of persistents you have in memory, check out your
#cachedObjectCountByClass.  You should have no more than one 19K tree
worth of objects given that you are doing a garbageCollect after each
one.

If you would share the skeleton of your load script it also might shed
some light.

Thanks,
  Chris


On 8/6/07, Burella Juan M. <juan.burella at gmail.com> wrote:
> I need to do a massive migration to Magma. My approach is currently like
> this: I'm loading the data from external files, then build a complex object
> and commit it to a magma repository, and over again until the file ends.
>
> While doing this, my image was growing at very high rates, approx. 200Mb and
> taking several days, between 5 or 6. So, to see what's happening, I've done
> some tests with toy objects, each one of them has 5 iVars. and each one has
> another 5 iVars. This generates a tree of 19.556 objects
> (5^0+5^1+5^2+5^3+5^4+5^5). Then I started to adding 50 of these "tree" (or
> smock?) objects, by groups of 5. I tried to maintain the complexity of these
> objects by adding "random" variable sized strings to the leafs and other
> miscellaneous objets.
> When this 50 are finalized, I've done:
>
> self magmaSession stubOut: aCollection. "aCollection has 50 smock objects"
> aCollection := nil.
> Smalltalk garbageCollect.
> self magmaSession finalizeOids.
>
> But still the image is increasing its size. At beggining of the test the
> image size (as reported from #vmParameterAt: 3) was approx. 60Mb and at the
> end was approx. 89MB.
>
> Any suggestions for reducing the growing of the image size?
>
> Thanks in advance.
> Juan Matias.
>
> _______________________________________________
> Magma mailing list
> Magma at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
>