When populating a MagmaCollection I got error with index, because I used the wrong selector for the index. I fix that, then I start to get "class definition unknow" error message, then my whole data is unaccessible (lost), then from the server I stop and restrat (even restarting Squeak), and now each time I start the Magma server I got the two following messages (capture here).
I am sure I do a lot of mistake when using Magma, but now I start to get anxious with it.
Hilaire
I recover from a backup, and now when I am trying to populate these new MagmaCollection I got a class definition unknow error.
In which direction should I look at? (enclose a capture of the error message)
Hilaire
Hilaire Fernandes a écrit :
When populating a MagmaCollection I got error with index, because I used the wrong selector for the index. I fix that, then I start to get "class definition unknow" error message, then my whole data is unaccessible (lost), then from the server I stop and restrat (even restarting Squeak), and now each time I start the Magma server I got the two following messages (capture here).
I am sure I do a lot of mistake when using Magma, but now I start to get anxious with it.
By investigating step by step, I have found the relevant piece of code which cause the error. It is related to the addition of a DateAndTime index.
My commit involves the creation of several related/nested MagmaCollection each coming with one index. The top bottom collection is created with a DateAndTime index:
IFIEtayageStepAchievement>>initialize super initialize. exercicesRecord := MagmaCollection new. exercicesRecord addIndex: (MaDateAndTimeIndex attribute: #endTime). cumulativeLearningGain := 0. failures := 0. successes := 0
If I remove the addIndex: it is ok.
I re-check my #endTime is really returning a DateAndTime object. At first #endTime was retrieving the object from other attribute object, now endTime is also an instance variable, but I think it does not really matter.
Are there any precaution when using DateAndTime index?
Hilaire
Hilaire Fernandes a écrit :
I recover from a backup, and now when I am trying to populate these new MagmaCollection I got a class definition unknow error.
In which direction should I look at? (enclose a capture of the error message)
Hilaire
Hilaire Fernandes a écrit :
When populating a MagmaCollection I got error with index, because I used the wrong selector for the index. I fix that, then I start to get "class definition unknow" error message, then my whole data is unaccessible (lost), then from the server I stop and restrat (even restarting Squeak), and now each time I start the Magma server I got the two following messages (capture here).
I am sure I do a lot of mistake when using Magma, but now I start to get anxious with it.
Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma
You are adding that DateAndTimeIndex in one of your existing databases right? Not creating from scratch, right? When I create from scratch DateAndTimeIndexes have no problem:
|path session | path _ 'c:\temp\DateAndTimeTest'. session _ MagmaSession openLocal: path. session connectAs: 'c'. mc _ MagmaCollection new addIndex: (MaDateAndTimeIndex attribute: #key) ; yourself. mc add: (DateAndTime now) -> 'hello'. session commit: [ session root at: 'mc1' put: mc ]. session
I think you are seeing already-corrupt data in your old database..?
So far the only way I've been able to corrupt a db is due to that bug with compress; working on that one..
--- Hilaire Fernandes hilaire2006@laposte.net wrote:
By investigating step by step, I have found the relevant piece of code which cause the error. It is related to the addition of a DateAndTime index.
My commit involves the creation of several related/nested MagmaCollection each coming with one index. The top bottom collection is created with a DateAndTime index:
IFIEtayageStepAchievement>>initialize super initialize. exercicesRecord := MagmaCollection new. exercicesRecord addIndex: (MaDateAndTimeIndex attribute: #endTime). cumulativeLearningGain := 0. failures := 0. successes := 0
If I remove the addIndex: it is ok.
I re-check my #endTime is really returning a DateAndTime object. At first #endTime was retrieving the object from other attribute object, now endTime is also an instance variable, but I think it does not really matter.
Are there any precaution when using DateAndTime index?
Hilaire
Hilaire Fernandes a écrit :
I recover from a backup, and now when I am trying to populate these
new
MagmaCollection I got a class definition unknow error.
In which direction should I look at? (enclose a capture of the
error
message)
Hilaire
Hilaire Fernandes a écrit :
When populating a MagmaCollection I got error with index, because
I
used the wrong selector for the index. I fix that, then I start to
get
"class definition unknow" error message, then my whole data is unaccessible (lost), then from the server I stop and restrat (even
restarting Squeak), and now each time I start the Magma server I
got
the two following messages (capture here).
I am sure I do a lot of mistake when using Magma, but now I start
to
get anxious with it.
------------------------------------------------------------------------
------------------------------------------------------------------------
Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma
Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma
Yep, I created a *clean* copy of the relevant MagmaCollection, I have to wrote some special code for that, but it could be usefull in future for cleaning backup purpose. Now adding the the DateAndTime index seems to be just fine. Now I will figure out to use such index. Thanks for the tips
Hilaire
Chris Muller a écrit :
You are adding that DateAndTimeIndex in one of your existing databases right? Not creating from scratch, right? When I create from scratch DateAndTimeIndexes have no problem:
|path session |
All,
In light of the problem Hilaire has uncovered, please don't use MagmaCompressor at this time.
I have confirmed a couple of issues with the compressor that are leaving certain classes it thinks are unreferenced behind (such as unreferenced superclasses of *referenced* subclasses), leading to the "class id not found".
I am working on this fix, some improvements, and a "Repository Map" tool that will print a detailed report to diagnose, and even fix some types of issues related to the repository files should it ever be needed. For example, the objects.idx file can be completely rebuilt from the "objects" file, if necessary (even though it never should be). Should anyone ever encounter this type of issue again, this tool will make the problem much clearer, particularly through e-mail.
I am also considering extending the use of the "commitPackages" file, which is a "transaction Log" of the commits since the last flush used for recovery in the event of a power-outage. Currently, every commit flushes the serialized MaCommitPackage to the commitPackages file and forks a 5-second timer after which the other repository files will be written to disk in the background (with appropriate Monitor guards, of course). After this flush, the commitPackages file is repositioned to the beginning where the next commit overwrites the previous commitPackages. I don't think it would take much to do like commercial databases and keep appending to the commitPackages file so it could even be applied against the last cold backup if necessary, but it might introduce some added complexity..
Given my schedule at the moment, all this might take a bit, thank you for your patience. In the mean time, please do not use MagmaCompressor.
Best, Chris
--- Hilaire Fernandes hilaire2006@laposte.net wrote:
Yep, I created a *clean* copy of the relevant MagmaCollection, I have to wrote some special code for that, but it could be usefull in future for cleaning backup purpose. Now adding the the DateAndTime index seems to be just fine. Now I will figure out to use such index. Thanks for the tips
Hilaire
Chris Muller a écrit :
You are adding that DateAndTimeIndex in one of your existing
databases
right? Not creating from scratch, right? When I create from
scratch
DateAndTimeIndexes have no problem:
|path session |
Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma
Hi Hilaire,
When populating a MagmaCollection I got error with index, because I used the wrong selector for the index.
Can you provide a lot more detail? What was the index type and what did you discover you did wrong? What error was reported and then how did you fix it? I ask because the next sentence implies this is what led to the problem..
Also, this is a development database right? You are writing new code with new indexes and accidently used the wrong selector. Anytime you are doing exploratory, volatile stuff like this, you definitely want to have a safe backup of your database if the data is important. Just like a Smalltalk image model can get goofed up, the db model CAN get goofed up during development.
But file corruption should NEVER happen, so that's why I'm very curious about the details of what led up to this problem. If there is a way for the files to be corrupted, even through misuse, I want to be sure to prevent that.
I fix that, then I start to get "class definition unknow" error message,
99% of the time, the root cause of this error is your client has extended the protocol beyond what the server expects. By looking in the right place in the debugger it will show which class belongs to the new id that the server could not interpret. First, look at what id it says is not found in the server's debugger, then explore your client MagmaSession which sent the request:
aMagmaSession -> link -> serializer -> classIdManager -> classesById
One possibility is you creaed a new index-type that was not present on the server, therefore not part of the servers protocol, so it couldn't materialize it..
then my whole data is unaccessible (lost),
I'll bet it's not lost. You can probably recover it using Magma's tools, but you will need to get familiar with them..
then from the server I stop and restrat (even restarting Squeak), and now each time I start the Magma server I got the two following messages (capture here).
This error is signaled when Magma is unable to materialize the repository's class definitions because of ANY Error. If you look at the method #materializeClassDefinitions, it will have captured the high-level exception Error. Can you please inspect the :err variable and see what error it encountered?
I am sure I do a lot of mistake when using Magma, but now I start to get anxious with it.
Well, thankfully the Magma file format relatively simple and extremely well-documented (thanks to Brent and his team). All of its "records" are encapsulated by first-class objects so you can easily "browse" or interrogate (via script) the contents of the database and see where some record has "bad data" and then fix it.
Contrast this with something like ImageSegments. Heaven forbid some sort of corruption in an ImageSegment which utilizes low-level bytes without first-class representations of its parts and, to my knowledge, is not well documented..
Magma does does have *some* checks because I knew people would be "rough" with it, but there are obviously some places that still need better checks (like letting the client extend the protocol). You have been very helpful in reporting these issues and finding some bugs (and the DNU in your screen shot is another bug you've found, I'll fix that).
If you must recover this file, check out pages 1-4 of the file-format documentation
http://wiki.squeak.org/squeak/uploads/2665/magma_file_format.pdf
and then e-mail me and I will help you.
Regards, Chris
Chris Muller a écrit :
Hi Hilaire,
When populating a MagmaCollection I got error with index, because I used the wrong selector for the index.
Can you provide a lot more detail? What was the index type and what did you discover you did wrong? What error was reported and then how did you fix it? I ask because the next sentence implies this is what led to the problem..
When adding a new MagmaCollection, I was adding an index with a wrong accessor name (#step instead of the correct one #stepNumber):
IFIEtayageAchievement>>initialize super initialize. etayageStepAchievements := MagmaCollection new. etayageStepAchievements addIndex: (MaIntegerIndex attribute: #stepNumber)
I fix that, then I start to get "class definition unknow" error message,
See the attached screen capture, it show the id in question is about Integer. Looking at the debugger, in instance variable classById, integer has two id 107 and 214. I don't know if this normal or not.
99% of the time, the root cause of this error is your client has extended the protocol beyond what the server expects. By looking in the right place in the debugger it will show which class belongs to the new id that the server could not interpret. First, look at what id it says is not found in the server's debugger, then explore your client MagmaSession which sent the request:
aMagmaSession -> link -> serializer -> classIdManager -> classesById
I will investigate that.
One possibility is you creaed a new index-type that was not present on the server, therefore not part of the servers protocol, so it couldn't materialize it..
No, it should not be that problem this time :)
Thanks,
Hilaire
Chris Muller a écrit :
I fix that, then I start to get "class definition unknow" error message,
99% of the time, the root cause of this error is your client has extended the protocol beyond what the server expects. By looking in the right place in the debugger it will show which class belongs to the new id that the server could not interpret. First, look at what id it says is not found in the server's debugger, then explore your client MagmaSession which sent the request:
aMagmaSession -> link -> serializer -> classIdManager -> classesById
Chris, this error does not pop up in the server, but in the client. I am talking about the 'MaObjectSerializationSoftwareError: class-definition not found' error.
With this error, nothing appear in the server. It is only when shutting down then restarting the server, I got the other error in the server (see my post with the two sreenshot. After this error the ODB cannot be re-used, I have to recover from back-up.
One possibility is you creaed a new index-type that was not present on the server, therefore not part of the servers protocol, so it couldn't materialize it..
Normally no. In case of doubt (because I modify one index method), I uploaded the newest class definition in the server, but still same problem. I think I have a problem in my new code, because some ancient version of it was working in November.
Thanks,
Hilaire
Hilaire, in testing your other issue with the compress ("MagmaSet has no id assigned") I have somehow reproduced the issues you describe here! Unfortunately I don't know for sure what I did but I think it involved saving the image with the debugger open in the middle of the server code.
As you may know, Magma restores the state (open, connected, etc.) of all servers and clients upon restarting the image, and it may be related to this combination of things..
I'm very intrigued into this problem now, I'll let you know what I find. (and, if you find additional information let me know..).
- Chris
--- Hilaire Fernandes hilaire2006@laposte.net wrote:
When populating a MagmaCollection I got error with index, because I used the wrong selector for the index. I fix that, then I start to get "class definition unknow" error message, then my whole data is unaccessible (lost), then from the server I stop and restrat (even restarting Squeak), and now each time I start the Magma server I got the two following messages (capture here).
I am sure I do a lot of mistake when using Magma, but now I start to get anxious with it.
Hilaire
Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma
magma@lists.squeakfoundation.org