Saving big objects?

[Sebastian Sastre]

Thanks for patiently educating me about this details Chris. Indeed making
transactions to commit in that medium-small size is the way to go (a very
reasonable tip for any transactional system by the way).

The part I have to understood better is how I will make that discard of
objects you mention to keep image not that hungry of RAM. I looked Gjallar
use of #stubOut: to see an usage reference of it but hasn't senders. What
are the best practices of using #stubOut:? I mean, I imagine what to do when
one has to traversal a big model already loaded in magma discarding parts as
the process goes ahead but what happen if one are creating that model like
in this load test? Of course this test are not realistic usability because
the real use will make lots of small changes making the model grow at ant
step. But supose you have an application that instantiates initially (on
first user hit for instance) making a magma repository to have a
significatively big standard repository and once created that small changes
will be made by the users. What if that instantiation of the model in magma
makes squeak go beyond 200 megabytes? As I saw it crash systematically about
250 - 300	megas I wonder how to keep things below 200.

	thanks again,

Sebastian Sastre


> -----Mensaje original-----
> De: Chris Muller [mailto:asqueaker at gmail.com] 
> Enviado el: Sábado, 20 de Octubre de 2007 12:45
> Para: Sebastian Sastre
> CC: magma at lists.squeakfoundation.org
> Asunto: Re: Saving big objects?
> 
> Hi Sebastian,
> 
> Magma can go plenty big, but the individual transactions 
> which build it up must, themselves, remain under 10meg each.  
> This is an arbitrary limit that you could change, but it is 
> really more polite to other users not to because it ensures 
> the servers time is sliced into smaller pieces so other users 
> can have a chance to get their work done too.
> 
> What you should do is commit every 100 Persons with 20 Acts 
> each instead of all 10000 at once.
> 
> Now, when you do this, be sure to discard the Persons you've 
> already committed from memory, otherwise you will be 
> accumulating a larger and larger portion of the 
> now-persistent model in your local client, which will bog 
> down subsequent commits.  See the performance tuning tips on 
> the Magma documentation pages for more information.
> 
>   http://wiki.squeak.org/squeak/2985
> 
> >     I've read about starting small with Magma but I need a 
> glimpse on 
> > how the model I'm thinking on support with Magma will 
> scale. From this 
> > test it seems that Magma collections supports lots of 
> objects but any 
> > common object suporting even a low number of magma collections that 
> > will be growing will easily reach more that 10 megabytes when 
> > serialized. So I'm missing something?
> 
> Yes.  MagmaCollections are tiny for the serializer, even if 
> they have a billion objects in them.  The serializer only 
> serializes new or changed objects, so you do need to do them 
> a smaller-than-10-meg piece at a time.
> 
> >     Same question in other scenario: the main object of the domain 
> > model the root of the application domain from where all is 
> accessed by 
> > reachability will also suffer this limitation. What one can 
> do at that point?
> 
> The serializer only serializes what is new or changed which 
> is completely independent of the overall size of the persistent model.
> 
> Regards,
>   Chris
> 
> On 10/20/07, Sebastian Sastre <ssastre at seaswork.com> wrote:
> >
> >
> > Hi Chris, all,
> >
> >     I've been reading some Gjallar code to see how they use 
> Magma. I 
> > saw the a main object which is tha application model and it 
> has 4 or 5 
> > MagmaCollections in it's instVars.
> >
> >     So I in a load test I've made a model object with a magma 
> > collection with 10000 person objects. It performed nicely.
> >
> >     Then I pushed that by making this person object to 
> store in one of 
> > it's instVars #acts and set up there a magma collection and 
> added 20 
> > acts objects to every person. So it should end up with 10K persons 
> > having one magma collection with 20 act objects each.
> >
> >     What I've got is a serializer security exception 
> > MaObjectSerializationSecurityViolation saying that one 
> can't serialize 
> > objects of more than 10 megabytes.
> >
> >     I've read about starting small with Magma but I need a 
> glimpse on 
> > how the model I'm thinking on support with Magma will 
> scale. From this 
> > test it seems that Magma collections supports lots of 
> objects but any 
> > common object suporting even a low number of magma collections that 
> > will be growing will easily reach more that 10 megabytes when 
> > serialized. So I'm missing something? There is a more 
> appropiate use of Magma for a model like that?
> >
> >     Same question in other scenario: the main object of the domain 
> > model the root of the application domain from where all is 
> accessed by 
> > reachability will also suffer this limitation. What one can 
> do at that point?
> >
> >     thanks,
> >
> >
> >
> > Sebastian Sastre
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Magma mailing list
> > Magma at lists.squeakfoundation.org
> > http://lists.squeakfoundation.org/mailman/listinfo/magma
> >
> >