...

...

Hi,

Hi Jerry,

...

...

I seem to have some sort of corruption in one of my Magma databases.

Incidentally, corruption can occur if, on a particular system, the Squeak call to StandardFileStream>>#flush does not guarantee, upon return, bits were indeed flushed to the platter and the system experiences a hard power-off.

If this happens and happened to cause a corruption, the easiest remedy is to restore from last full backup, then roll-forward to the second before the power-out.

But the type of corruption you have seems to be related to the various history of instance migrations of STBUser, not a power-outage.

...

...

I'm running a Seaside application with Magma 0.41gamma1 and

I hope you meant 42, not 41.

...

...

I have 2 test images and 2 databases.   Both images can access one of the databases with no problems.  Trying to access the other database from either image results in a error.   So I assume that there is something wrong with the "bad" database and not a problem with my images.

Please clarify whether you are connecting with brand new MagmaSession instances in each case, as opposed to an old session.

...

...

The error occurs during initial startup,  beginning with:

BTW, the stack-trace attachment is always helpful.

...

...

Index: 1 Version: 4 ivar count: 5 Index: 2 Version: 5 ivar count: 6 Index: 3 Version: 2 ivar count: 9 Index: 4 Version: 1 ivar count: 10 Index: 5 Version: 3 ivar count: 11

Hm, that shouldn't happen.

...

...

The instance variable counts look right, but the version numbers appear

Would you clarify what you mean by, "The instance variable counts look right"?

...

...

to be wrong.  Also, the inImageSerializer believes that we are on version 3:

def := (MaObjectSerializer allInstances first classIdManager inImageDefinition: STBUser). def version -> 3 def namedInstSize -> 11 (which is the correct number)

Great information, thanks!

...

...

So it appears that there are too many versions in Magmas' class definition collection for my class, and those versions are out of order.

You absolutely don't have "too many", your image may run with any "version" you want. Magma simply maps the instance-variables by name. There is no inherent requirement for Magma to be running at any particular version in the image, but obviously an image running with fewer inst-vars may truncate data from buffers committed by images running with more. Magma issues a warning in that case.

...

...

Somehow I always manage to make interesting errors!

How did you stumble into this thicket?

...

...

Also: after re-arranging those class definitions and closing the Magma session, closing my image without saving, and restarting the image, my application opened with no problems.   So I guess my changes to the Magma class definitions were persisted.   However, that does leave me with version 4 and 5 definitions; while the in-image version of STBUser is 3.   I'm sure I could go in and remove those definitions, and close the session and everything will be OK.

Please don't remove any of the definitions. If you still have an instances of that version in the repository the session will blow up when trying to materialize them.

...

...

So my questions are:

Any idea how this could have possibly happened?

No, but if you are running a repository with buffers with five different versions of the same class, it is necessary to understand the details of how Magma handles all scenarios. It is worth documenting somewhere your process for evolving legacy instances from 5 to 11 inst-vars. Having 5 versions of the same class of instances in the database is common during development, but rarer in production. Production systems ensure instance migrations are handled deliberately and with some care.

...

...

How can I fix it permanently, preferably through a code patch and not manual "live" tinkering?

Option 1: The quickest and easiest fix was at the top of the e-mail; restore from backup and roll forward to just before the commit that caused the corruption (You can use the MagmaBufferBrowser to determine the commitNumber of the mis-ordered OC, then look at the time-stamp. You can roll-forward to just before that.

Option 2: What matters is that STBUser buffers in the repository all point consistently to the correct class version. Write a script using MagmaFileTraverser to interrogate all of the Fixed buffers ("type = 3") where classId=126, check that their versions all are correct and consistent with their number of instVars of the Buffer (#objectInstSize).

If they are all consistent, then connect a fresh MagmaSession to the database and commit your change to the mis-ordered OC. Stop and dispose of all other running sessions, replace them with fresh ones.

...

...

Is there any way to make sure it does not happen again?

Consider committing a change to all of the classDefinitionsById to a Dictionary of *SortedCollections* instead of OrderedCollection. The sortBlock would be:

[ : a : b | a version < b version ]

- Chris

Hi,

I started again with another copy of the "bad" database directory, so there have been no changes to the MaClassDefinitions.

I was able to get a MagmaFileTraverser connected and running.

I have 9 STBUser instances in my database.  7 claim to be version 4 with 10 instance variables, 2 claim to be version 1 with 10 instance variables.  I believe all should be version 4 with 10 instance variables.   My  current image class definition has 10 ivars.

Since only one of your MaClassDefinitions has 10 inst-vars, any buffer of that class that also has 10 instVars should be pointing to that MaClassDefinition. i.e., it sounds like the two that claim to be version 1 are pointing to the incorrect version.

I believe the image thinks that our current in-image version is 1 because that is the version number of the last entry in the MaClassDefinitions for STBUser.

The image uses whichever one matches the #allInstVarNames defined in the image.

I think I should change the version numbers of each STBUser MaClassDefinition so they are in order from 1 to 4, but I don't need to re-order the definitions.

Yes, that sounds right based on the information given.

This will mean that new instances of STBUser will be version 4, which is correct.

New instances will be based on the inImageDefinition, which is based on the allInstVarNames defined to STBUser in the image that instantiates the new STBUser.

Finally, I should go through the 2 incorrect instances on disk and convert their class ID to 4 from 1.

Yes.

Is this something that MagmaFileTraverser can handle?  I can change the buffer's values, but can I write the repaired buffer back to disk?

No, the only way buffers get changed in a Magma repository is via a commit through a MagmaSession, because you want the repository to remain consistent with the commit-log.

All you need to do is commit those two objects that appear to be pointing to the wrong version of the class. Pull them up by oid, assign them to a temp var so no chance of GC, commit a change to them, then commit another change back to the original value. Whatever version of the class is loaded in the image is the version that will be written to the buffer (so verify ahead of time the image definition of STBUser matches version 4!).

You can then use the FileTraverser again or the MagmaBufferBrowser to verify the original buffers now point to the correct class-definition version.

- Chris

Hi all,

Let me preface this by saying that the problems I've been seeing are almost certainly due to something I've done horribly wrong, and it's likely that these changes won't be useful for anyone else. If they are useful to someone, they will require some tweaking for your specific problem.

Regardless, here is a fix that seems to work for me.

Attached are 2 changesets - DBHackFix and NoOverwriteVersion.

Here's how I fixed my database:

1) Started an image and loaded DBHackFix - this let me open my database successfully. 2) Ran the following code, which fixes my instances on disk.

session := MagmaSession openLocal: 'magma'. session connectAs: 'test'.

root := session root.

session begin. serializer := session serializer. classIdManager := serializer classIdManager. defs := classIdManager classDefinitionsById at: 126. defs doWithIndex: [:each :index | each version: index]. session commit. session disconnectAndClose.

3) Closed the image without saving

At this point, the database is consistent and everything works fine, until I start up my application and try to add a new instance of STBUser and restart the image. Then everything breaks again. It seemed as if the class definition versions for STBUser were being set wrong. (All other classes in the database seem to be fine).

So, I tried to track down what was happening by changing MaClassDefinition>>version to this:

version: anInteger

(id = 126) ifTrue:[self halt]. version _ anInteger

I'm not sure exactly what is happening or why, but it seems as if the first class definition of STBUser encountered in my database is version 4, which is the latest in the image. However in MaClassIdManager>>addClassDefinition: the version is then set to the current number of known STBUser class definitions, which is 1. So, I changed the version: method in MaClassDefinition to ignore the new version, unless the current version is nil or zero, which are needed when a new class definition is created. I also changed MaClassIdManager>>ensureClassDefinitionsFor: so that a SortedCollection sorted by version number is used for the class defs, instead of an OrderedCollection (as suggested by Chris). These changes are in the NoOverwriteVersion changeset.

As I said, I'm still not sure of what the root cause of the problem is. I don't think there are other instances of STBUser in my database with other class def version numbers, if there were I would have expected the MagamFileTraverser code would have seen them. Perhaps all class definitions are stored somewhere in the database, and somehow the definitions for my STBUser ended up out of order?

Regardless, next step is:

4) Load NoOverwriteVersion changeset into the image, and save the image. 5) Start the image again. Start application as normal. Everything should work OK from here on.

Surprisingly enough this seems to work fine, I've been able to add new STBUser instances as well as other objects, and I've been able to add two instance variables to STBUser along with new instances after each instance variable is added.

============

I'd like to thank Chris for his help and of course all of his work on Magma. It's a great system and it was nice to be able to use the usual Smalltalk tools to track down and fix my problem, even though it was a pretty low level issue and was a little strange. These sorts of experiences only increase my confidence in Magma.

Thanks,

Jerry

Hi,

You might be interested in trying the hack I used. No guarantees, and you should make sure you have good backups etc.

Basically it tries to catch errors such as the ones we've run into and uses the in-image definition for loading the objects from disk.

I'm sure there are many ways this can break, but it might work for what you need. I'm using it as a temporary part of a process that will hopefully fix my issue, I don't plan to keep it loaded in my images once my database is repaired.

In MaFixedObjectBuffer>>createObjectUsing: try something like this:

skeleton _ indexedSize = 0 ifTrue: [ class basicNew ] ifFalse: [ [class basicNew: indexedSize] on: Error do: [sourceDefinition := inImageDefinition. indexedSize := self objectInstSize. class basicNew] ].

-Jerry

On May 11, 2009, at 3:03 PM, Hernán Morales Durand wrote:

2009/5/7 Chris Muller ma.chris.m@gmail.com:

...

Option 2: What matters is that STBUser buffers in the repository all point consistently to the correct class version. Write a script using MagmaFileTraverser to interrogate all of the Fixed buffers ("type = 3") where classId=126, check that their versions all are correct and consistent with their number of instVars of the Buffer (#objectInstSize).

I have a similar problem ( 'DateQualifier cannot have variable sized instances' ) but in my case there's only one version of DateQualifier, ( i.e. aMagmaClassIdManager definitionForClassId: 378 version: 2 fails because non-existent index). Here objectInstSize = 2 and sourceDefinition namedInstSize = 3, resulting in indexedSize = -1 and performing the #basicNew:

#namedInstSize is correct, the DateQualifier's superclass has 3 instance variables, the output of Jerry's first script would be:

Index: 1 Version: 1 ivar count: 3

So I wrote the script you suggested:

( MagmaFileTraverser for: 'Magma' ) open anchorGraphDo: [ : buffer | ( buffer classId = 378 and: [ buffer class = MaFixedObjectBuffer ] ) ifTrue: [ buffer objectInstSize inspect ] ].

but still fails with the same error. My problem is that I cannot open the repository to commit any corrections because of this error.

11 May 2009 4:47:59 pm VM: Win32 - a SmalltalkImage Image: Squeak3.10.2 [latest update: #7179]

SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir C:\Squeak 3.9 Trusted Dir C:\Squeak 3.9\hmorales Untrusted Dir C:\My Squeak\hmorales

DateQualifier class(Object)>>error: Receiver: DateQualifier Arguments and temporary variables: aString: 'DateQualifier cannot have variable sized instances' Receiver's instance variables: superclass: AbstractTitleQualifier methodDict: a MethodDictionary(#distantGetter->a CompiledMethod (720) #initiali...etc... format: 136 instanceVariables: nil organization: ('printing' printOn: printQualifierOn:) ('initialization' distant...etc... subclasses: nil name: #DateQualifier classPool: nil sharedPools: nil environment: a SystemDictionary(lots of globals) category: #'DocModel-Titles'

DateQualifier class(Behavior)>>basicNew: Receiver: DateQualifier Arguments and temporary variables: sizeRequested: -1 Receiver's instance variables: superclass: AbstractTitleQualifier methodDict: a MethodDictionary(#distantGetter->a CompiledMethod (720) #initiali...etc... format: 136 instanceVariables: nil organization: ('printing' printOn: printQualifierOn:) ('initialization' distant...etc... subclasses: nil name: #DateQualifier classPool: nil sharedPools: nil environment: a SystemDictionary(lots of globals) category: #'DocModel-Titles'

MaFixedObjectBuffer>>createObjectUsing: Receiver: a MaFixedObjectBuffer oid : 16861039 classId : 378 objectInstSize : 2 commitNumber : ...etc... Arguments and temporary variables: aMaObjectSerializer: a MaObjectSerializer class: DateQualifier skeleton: nil indexedSize: -1 sourceDefinition: a MaClassDefinition DateQualifier version = 1 inImageDefinition: a MaClassDefinition DateQualifier version = 1 eachOid: nil sourceIndex: nil targetIndex: nil Receiver's instance variables: byteArray: a ByteArray(242 159 4 0 176 14 62 0 0 0 62 0 0 19 115 0 99 81 1 0 0 ...etc... startPos: 299

MaObjectSerializer>>newSkeletonFor:fromProxy: Receiver: a MaObjectSerializer Arguments and temporary variables: aMaObjectBuffer: a MaFixedObjectBuffer oid : 16861039 classId : 378 objectI...etc... aMaMutatingProxy: nil skel: nil Receiver's instance variables: oidManager: a MagmaOidManager classIdManager: a MagmaClassIdManager preMadeObjectBuffers: an Array(a MaByteObjectBuffer oid : 16149077336050 cla...etc... graphBuffer: a MaSerializedGraphBuffer (6168 objects in 303090 bytes) createProxyBlock: [] in MagmaSession>>initializeSerializer {[:oid | MagmaMutati...etc... traversalStrategy: a MaObjectGraphTraversalStrategy amBusy: true truncatedObjects: a WeakIdentityKeyDictionary() preferences: a MaObjectSerializerPreferences bpMapIndex: 1

--- The full stack --- DateQualifier class(Object)>>error: DateQualifier class(Behavior)>>basicNew: MaFixedObjectBuffer>>createObjectUsing: MaObjectSerializer>>newSkeletonFor:fromProxy:

---

[] in MaObjectSerializer>>materializeGraphDo: {[skeletons add: (Association key: (newObject := self newSkeletonFor: each ...]} [] in MaWeakValueDictionary(MaDictionary)>>maAt:ifPresent:ifAbsent: {[^ zeroArgBlock value]} MaWeakValueDictionary(MaDictionary)>>at:ifAbsent: MaWeakValueDictionary(MaDictionary)>>maAt:ifPresent:ifAbsent: [] in MagmaOidManager>>objectWithOid:ifFound:ifAbsent: {[(MaOidCalculator isOidForNewObject: anInteger) ifTrue: [newObjects ma...]} MaOidCalculator class>>objectWithOid:ifNone: MagmaOidManager>>objectWithOid:ifFound:ifAbsent: MaObjectSerializer>>objectWithOid:ifFound:ifAbsent: [] in MaObjectSerializer>>materializeGraphDo: {[:each | object := self objectWithOid: each oid ifFound: [:foundObj...]} [] in MaSerializedGraphBuffer>>buffersDo: {[:eachPosition | buffer := preMadeBuffers at: (self bufferTypeAt: eachP...]} MaSerializedGraphBuffer>>bufferPositionsDo: MaSerializedGraphBuffer>>buffersDo: MaObjectSerializer>>materializeGraphDo: MaObjectSerializer>>materializeGraph:do: [] in MagmaSession>>materializeObject: {[serializer materializeGraph: aByteArray do: [:eachObject :eachBuffer | ...]} ...etc...

Any other suggestion?

Hernán _______________________________________________ Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma

2009/5/12 Chris Muller ma.chris.m@gmail.com:

Hi Hernán, this could mean two clients had different notions of what class 378 is.  I would use the FileTraverser to take a count of your 378's; see first if you have consistency.

Hi Chris, I forgot to mention, 378 is DateQualifier. Should I search for more DateQualifier class definitions in the aMagmaClassIdManager ?

Do you have any suspicions about the cause of the problem?

some questions:

• What version of Magma?

release 42 gamma 1 (Magma client-cmm.398)

• Is DateQualifier a kind-of Set?

No, is a subclass of AbstractTitleQualifier which is a subclass of Object.

• Did you extend (subclass) any class included in the MagmaSession protocol?

No

Hernán

- Chris

On Mon, May 11, 2009 at 3:03 PM, Hernán Morales Durand hernan.morales@gmail.com wrote:

...

2009/5/7 Chris Muller ma.chris.m@gmail.com:

...

Option 2:  What matters is that STBUser buffers in the repository all point consistently to the correct class version.  Write a script using MagmaFileTraverser to interrogate all of the Fixed buffers ("type = 3") where classId=126, check that their versions all are correct and consistent with their number of instVars of the Buffer (#objectInstSize).

I have a similar problem ( 'DateQualifier cannot have variable sized instances' ) but in my case there's only one version of DateQualifier, ( i.e. aMagmaClassIdManager definitionForClassId: 378 version: 2 fails because non-existent index).  Here objectInstSize = 2 and sourceDefinition namedInstSize = 3, resulting in indexedSize = -1 and performing the #basicNew:

#namedInstSize is correct, the DateQualifier's superclass has 3 instance variables, the output of Jerry's first script would be:

Index: 1 Version: 1 ivar count: 3

So I wrote the script you suggested:

( MagmaFileTraverser for: 'Magma' ) open        anchorGraphDo: [ : buffer |                ( buffer classId = 378 and: [ buffer class = MaFixedObjectBuffer ] )                        ifTrue: [ buffer objectInstSize inspect ]                ].

but still fails with the same error. My problem is that I cannot open the repository to commit any corrections because of this error.

11 May 2009 4:47:59 pm VM: Win32 - a SmalltalkImage Image: Squeak3.10.2 [latest update: #7179]

SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir C:\Squeak 3.9 Trusted Dir C:\Squeak 3.9\hmorales Untrusted Dir C:\My Squeak\hmorales

DateQualifier class(Object)>>error:        Receiver: DateQualifier        Arguments and temporary variables:                aString:        'DateQualifier cannot have variable sized instances'        Receiver's instance variables:                superclass:     AbstractTitleQualifier                methodDict:     a MethodDictionary(#distantGetter->a CompiledMethod (720) #initiali...etc...                format:         136                instanceVariables:      nil                organization:   ('printing' printOn: printQualifierOn:) ('initialization' distant...etc...                subclasses:     nil                name:   #DateQualifier                classPool:      nil                sharedPools:    nil                environment:    a SystemDictionary(lots of globals)                category:       #'DocModel-Titles'

DateQualifier class(Behavior)>>basicNew:        Receiver: DateQualifier        Arguments and temporary variables:                sizeRequested:  -1        Receiver's instance variables:                superclass:     AbstractTitleQualifier                methodDict:     a MethodDictionary(#distantGetter->a CompiledMethod (720) #initiali...etc...                format:         136                instanceVariables:      nil                organization:   ('printing' printOn: printQualifierOn:) ('initialization' distant...etc...                subclasses:     nil                name:   #DateQualifier                classPool:      nil                sharedPools:    nil                environment:    a SystemDictionary(lots of globals)                category:       #'DocModel-Titles'

MaFixedObjectBuffer>>createObjectUsing:        Receiver: a MaFixedObjectBuffer        oid : 16861039        classId : 378        objectInstSize : 2        commitNumber : ...etc...        Arguments and temporary variables:                aMaObjectSerializer:    a MaObjectSerializer                class:  DateQualifier                skeleton:       nil                indexedSize:    -1                sourceDefinition:       a MaClassDefinition DateQualifier        version = 1                inImageDefinition:      a MaClassDefinition DateQualifier        version = 1                eachOid:        nil                sourceIndex:    nil                targetIndex:    nil        Receiver's instance variables:                byteArray:      a ByteArray(242 159 4 0 176 14 62 0 0 0 62 0 0 19 115 0 99 81 1 0 0 ...etc...                startPos:       299

MaObjectSerializer>>newSkeletonFor:fromProxy:        Receiver: a MaObjectSerializer        Arguments and temporary variables:                aMaObjectBuffer:        a MaFixedObjectBuffer        oid : 16861039        classId : 378        objectI...etc...                aMaMutatingProxy:       nil                skel:   nil        Receiver's instance variables:                oidManager:     a MagmaOidManager                classIdManager:         a MagmaClassIdManager                preMadeObjectBuffers:   an Array(a MaByteObjectBuffer        oid : 16149077336050        cla...etc...                graphBuffer:    a MaSerializedGraphBuffer  (6168 objects in 303090 bytes)                createProxyBlock:       [] in MagmaSession>>initializeSerializer {[:oid | MagmaMutati...etc...                traversalStrategy:      a MaObjectGraphTraversalStrategy                amBusy:         true                truncatedObjects:       a WeakIdentityKeyDictionary()                preferences:    a MaObjectSerializerPreferences                bpMapIndex:     1

--- The full stack --- DateQualifier class(Object)>>error: DateQualifier class(Behavior)>>basicNew: MaFixedObjectBuffer>>createObjectUsing: MaObjectSerializer>>newSkeletonFor:fromProxy:  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - [] in MaObjectSerializer>>materializeGraphDo: {[skeletons   add: (Association key: (newObject := self newSkeletonFor: each ...]} [] in MaWeakValueDictionary(MaDictionary)>>maAt:ifPresent:ifAbsent: {[^ zeroArgBlock value]} MaWeakValueDictionary(MaDictionary)>>at:ifAbsent: MaWeakValueDictionary(MaDictionary)>>maAt:ifPresent:ifAbsent: [] in MagmaOidManager>>objectWithOid:ifFound:ifAbsent: {[(MaOidCalculator isOidForNewObject: anInteger)   ifTrue: [newObjects    ma...]} MaOidCalculator class>>objectWithOid:ifNone: MagmaOidManager>>objectWithOid:ifFound:ifAbsent: MaObjectSerializer>>objectWithOid:ifFound:ifAbsent: [] in MaObjectSerializer>>materializeGraphDo: {[:each |  object := self     objectWithOid: each oid     ifFound: [:foundObj...]} [] in MaSerializedGraphBuffer>>buffersDo: {[:eachPosition |  buffer := preMadeBuffers     at: (self bufferTypeAt: eachP...]} MaSerializedGraphBuffer>>bufferPositionsDo: MaSerializedGraphBuffer>>buffersDo: MaObjectSerializer>>materializeGraphDo: MaObjectSerializer>>materializeGraph:do: [] in MagmaSession>>materializeObject: {[serializer materializeGraph: aByteArray   do: [:eachObject :eachBuffer | ...]} ...etc...

Any other suggestion?

Hernán