[Q]Populating magma with a large tree

Chris Muller afunkyobject at yahoo.com
Wed Sep 3 14:55:20 UTC 2003


--- Martin Drautzburg <martin.drautzburg at web.de> wrote:
> Chris Muller <afunkyobject at yahoo.com> writes:
>
> > A MagmaCollection is just like any other Collection in terms of referencing
> > objects, so yes, it is in there.  If you added your node to an
> > OrderedCollection then it would be in your Tree as well as the
> > OrderedCollection.  Removing the node from your Tree would not remove it
> from
> > the OrderedCollection, and it would not be gc'd, since it is still
> referenced
> > by the OrderedCollection.
> 
> For a normal Collection I understand this. But a MagmaCollection is
> nothing I would want to hold in memory. 

You need the MagmaCollection to support the indexed access you mentioned you
wanted.  There is no memory-resource reason to not hold a MagmaCollection in
memory since it only consumes one "page" of objects at a time, regardless of
its size.

> Suppose I have a million tree nodes and all of them are in the
> AllNodes MagmaCollection. I remove all references in memory, so I have
> zero nodes in memory. Then I traverse the tree from Magma root until I
> hit a leaf. If I have to go 6 levels I will have a minimum 6 Nodes in
> memory by then. I didn't touch the AllNodes MagmaCollection so far.

Once you know you're done with a node, you can tell your session to stubOut:
that node.

  mySession stubOut: myNode

This method uses become: as well as removeKey:, which can be slow in Squeak, so
you need to chop as high up and as infrequently as possible.  For example if,
after traversing the six nodes you're done with them, you could just stubOut:
the root node, and all that it reference will no longer be referenced.  I
wouldn't recommend stubbing after every node encountered since it might be
slow.

> When I remove the leaf from the AllNodes Collection, how many nodes
> will materialize ? None (because the only node I need is already in
> memory) or a million ?

None.  But I thought you were using the MagmaCollection so you could find any
node by some index key.  So you're only removing them from AllNodes when you
remove it from your graph, correct?

Keep in mind that MagmaCollections keep one "page" (one or two hundred, I don't
have the code in front of me at the moment to check exactly) of its objects in
memory at any one time.  You can customize this (see #pageSize:).  As you
access parts of the collection that are outside of the in-memory page, the new
page is brought in automatically and the old one is discarded allowing memory
to be reclaimed (assuming your program isn't referencing the objects in the old
page through some other in-memory object).

> What happens if I ask AllNodes size. Will any nodes materialize ?

No.

To sum up, in terms of "referencing", MagmaCollections act like
OrderedCollections.  It's just that they don't take up memory for all objects
they reference at a time.

Does this help?

 - Chris

__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com



More information about the Squeak-dev mailing list