Chris Muller a écrit :

...

                                    IFIModels users add: user].

Well, there you go. Isn't this referencing the entire, ever-growing database all in memory?

Really? I agree about the idea an of ever-growing database but IFIModels users just return the #users branch of the database, which is a MagmaCollection. Nothing special there.

What is special is that the added user objects contain itself two indexed Magma collections, and the problem is related to that because if I remove this part there are no problem.

Do you have test covering this case usage?

I try to read your test, not very easy to follow.

Hilaire

"You gotta let go, Luke." ;-)

Having said that, I am still experiencing some difficulty with memory "leaks" myself due to old stale MethodContexts not letting go of various references. It seems to be more of a Squeak problem that I don't fully understand yet.

...

Now I am wondering if creating empty indexed Magma collection could be a problem or should the collection be populated first?

No, either way should be no problem, and the test cases cover all the combinations IIRC. Look at #testMajorFunctions and #testAddIndexAndObjectsSimultaneously.

• Chris

On Jan 4, 2008 1:00 PM, Hilaire Fernandes hilaire@ofset.org wrote:

...

I was wondering if the index should only be created after the collection is in the database. So I changed the test, with two commit: blocks and the learner object's magma collections and indexes created at the second commit.

testAddLearner | user | 100 timesRepeat: [ IFIDbSession commit: [user := IFILearner new. IFIModels users add: user]. IFIDbSession commit: [user initLearningStates ; initPedagogicalMoments ]]. self assert: IFIMagmaResource current school learners size = 100

But the result is the same.

Now I am wondering if creating empty indexed Magma collection could be a problem or should the collection be populated first?

Hilaire

Hilaire Fernandes a écrit :

...

While writing test I met the following problem: I want to add hundred of (learner) object in a users MagmaCollection (it is located in a branch of the DB and it has index too)

While looping, the commit becomes slower and slower until Squeak stall at the 90th iteration. I know I can move the commit: in the outer block, it is faster but still hang at the final commit time. The mouse icon is then stuck to the write icon and the Squeak has to be brutally stopped.

In the Learner initialization if I remove the addIndex:, the process run at a constant speed and finish. It looks like a problem related to index creation.

testAddLearner | user | 100 timesRepeat: [user := IFILearner new. IFIDbSession commit: [IFIModels users add: user "IFIMagmaResource current school addPerson: user"]] self assert: IFIMagmaResource current school learners size = 100

IFILearner>>initialize super initialize. schoolYear := #CE2. learningStates := MagmaCollection new. learningStates "addIndex: (MaCompetencyIndex attribute: #competency);" addIndex: (MaFloatIndex attribute: #acquisitionLevel)

---

Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma

---

---

Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma

Hi Hilaire, from your first post, you wrote:

In the Learner initialization if I remove the addIndex:, the process run at a constant speed and finish.

Then the initialization code with, presumably, the way that does NOT exhibit the problem:

IFILearner>>initialize super initialize. schoolYear := #CE2. learningStates := MagmaCollection new. learningStates "addIndex: (MaCompetencyIndex attribute: #competency);" addIndex: (MaFloatIndex attribute: #acquisitionLevel)

If this is the case, please review your MaCompetencyIndex. It sounds like it's #indexHashesForIndexObject: may be answering progressively more "keys" to index by.

Multi-key indexing in the server, despite several good optimisations, over the years, is a resource intensive operation. If you try to index, say, 10000 keys in one commit, you would not see that fact reflected in the

'objects'

instance-variable of MaCommitPackage. Instead you will find it by exploring the commitPackage's

'allLargeCollectionChanges'

variable to see all the changes, including all the integral key values for the server-indexes. If as many as 10000, that commit will take a long time.

So yes, it is very good to have the commit in the inner loop. :)

=======

If you do not find any problem with your MaCompetencyIndex, here is a template script I put together based on my interpretation of the problem:

|mc subMc| mc := MagmaCollection new addIndex: (MaIntegerIndex attribute: #key); yourself. subMc := MagmaCollection new addIndex: (MaIntegerIndex attribute: #key); yourself. subMc add: 110 ->'subMc'. mc add: (100->subMc). MagmaRepositoryController create: 'c:\temp\hilaireTest' root: mc

It commits nested MagmaCollections, each with an index. Since I do not have access to your model, perhaps we can work together from this simple generic example to demonstrate the issue.

- Chris

On Jan 6, 2008 5:08 PM, Hilaire Fernandes hilaire@ofset.org wrote:

From the commit:[] of my testcase, I have checked the prepared MaCommitPackage instances, its objects size is constant at 30 objects/1137 bytes, even the one just before the hang. The hang happens in :

    commitResult := self submit: ((MaWriteRequest new)
                                            package: commitPackage;
                                            beginAnother: aBoolean;
                                            yourself)

and this loop does not finish:

aMagmaRepositoryRequest waitCursor showWhile: [link submit: aMagmaRepositoryRequest]

I am pretty sure the problem is connected with the creation of indexes of the MagmaCollection.

Hilaire

Hilaire Fernandes a écrit :

...

Chris Muller a écrit :

...

...

                                    IFIModels users add: user].

Well, there you go. Isn't this referencing the entire, ever-growing database all in memory?

Really? I agree about the idea an of ever-growing database but IFIModels users just return the #users branch of the database, which is a MagmaCollection. Nothing special there.

What is special is that the added user objects contain itself two indexed Magma collections, and the problem is related to that because if I remove this part there are no problem.

Do you have test covering this case usage?

I try to read your test, not very easy to follow.

Hilaire

---

Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma

Ah, ok. Your trouble can be solved by reading this message:

http://lists.squeakfoundation.org/pipermail/magma/2007-November/000895.html

- Chris

PS - You may be tempted to load newer versions of "Ma special collections" than 85, but 85 is the one that is still officially sanctioned as the "stable" version. 90+ are not (yet).

On Jan 8, 2008 4:27 AM, Hilaire Fernandes hilaire@ofset.org wrote:

Le lundi 07 janvier 2008 à 21:30 -0500, Chris Muller a écrit :

...

Hi Hilaire, from your first post, you wrote:

...

In the Learner initialization if I remove the addIndex:, the process run at a constant speed and finish.

Then the initialization code with, presumably, the way that does NOT exhibit the problem:

IFILearner>>initialize super initialize. schoolYear := #CE2. learningStates := MagmaCollection new. learningStates "addIndex: (MaCompetencyIndex attribute: #competency);" addIndex: (MaFloatIndex attribute: #acquisitionLevel)

If this is the case, please review your MaCompetencyIndex. It sounds

Sorry to be unclear about my code sample, the maCompetencyIndex is never used. And the hang occure when using standard indexes. The more there are indexes the quicker the hang.

From you example, and adapted to my use case I added a loop to stack some object:

I can stack 120 objects just fine. If I stack 150 objects it hangs, and currently no OID/indexes files appear in the DB folder, just the commits, object*, applied.* files and folder

|mc subMc| mc := MagmaCollection new addIndex: (MaIntegerIndex attribute: #key); yourself. 1 to: 150 do: [:i | subMc := MagmaCollection new addIndex: (MaIntegerIndex attribute: #key); yourself. subMc add: i ->'subMc'. mc add: ((i +150)->subMc)]. Transcript show: ' Done '. MagmaRepositoryController create: '/tmp/hilaireTest' root: mc

Hope we can find the issue

hilaire

---

Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma