Saving big objects?

[Chris Muller]

Hi Sebastian,

Magma can go plenty big, but the individual transactions which build
it up must, themselves, remain under 10meg each.  This is an arbitrary
limit that you could change, but it is really more polite to other
users not to because it ensures the servers time is sliced into
smaller pieces so other users can have a chance to get their work done
too.

What you should do is commit every 100 Persons with 20 Acts each
instead of all 10000 at once.

Now, when you do this, be sure to discard the Persons you've already
committed from memory, otherwise you will be accumulating a larger and
larger portion of the now-persistent model in your local client, which
will bog down subsequent commits.  See the performance tuning tips on
the Magma documentation pages for more information.

  http://wiki.squeak.org/squeak/2985

>     I've read about starting small with Magma but I need a glimpse on how
> the model I'm thinking on support with Magma will scale. From this test it
> seems that Magma collections supports lots of objects but any common object
> suporting even a low number of magma collections that will be growing will
> easily reach more that 10 megabytes when serialized. So I'm missing
> something?

Yes.  MagmaCollections are tiny for the serializer, even if they have
a billion objects in them.  The serializer only serializes new or
changed objects, so you do need to do them a smaller-than-10-meg piece
at a time.

>     Same question in other scenario: the main object of the domain model the
> root of the application domain from where all is accessed by reachability
> will also suffer this limitation. What one can do at that point?

The serializer only serializes what is new or changed which is
completely independent of the overall size of the persistent model.

Regards,
  Chris

On 10/20/07, Sebastian Sastre <ssastre at seaswork.com> wrote:
>
>
> Hi Chris, all,
>
>     I've been reading some Gjallar code to see how they use Magma. I saw the
> a main object which is tha application model and it has 4 or 5
> MagmaCollections in it's instVars.
>
>     So I in a load test I've made a model object with a magma collection
> with 10000 person objects. It performed nicely.
>
>     Then I pushed that by making this person object to store in one of it's
> instVars #acts and set up there a magma collection and added 20 acts objects
> to every person. So it should end up with 10K persons having one magma
> collection with 20 act objects each.
>
>     What I've got is a serializer security exception
> MaObjectSerializationSecurityViolation saying that one
> can't serialize objects of more than 10 megabytes.
>
>     I've read about starting small with Magma but I need a glimpse on how
> the model I'm thinking on support with Magma will scale. From this test it
> seems that Magma collections supports lots of objects but any common object
> suporting even a low number of magma collections that will be growing will
> easily reach more that 10 megabytes when serialized. So I'm missing
> something? There is a more appropiate use of Magma for a model like that?
>
>     Same question in other scenario: the main object of the domain model the
> root of the application domain from where all is accessed by reachability
> will also suffer this limitation. What one can do at that point?
>
>     thanks,
>
>
>
> Sebastian Sastre
>
>
>
>
>
>
> _______________________________________________
> Magma mailing list
> Magma at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
>