Hi:
I recently added support to load images that have been saved with the CogVM in image format 6505 to the RoarVM. However, now I noticed that the CogVM/StackVM crash when they are asked to load an image which is saved by our RoarVM.
Interestingly, the Squeak 4.2.1 and earlier are able to load this image.
To reproduce the problem, I put online an image [1] which is a stock Squeak4.2-10966, just downloaded, opened and saved with the RoarVM.
The Cog/StackVM fail to open it with an error in "0 org.squeakfoundation.Squeak 0x00033462 readImageFromFileHeapSizeStartingAt + 3058"
Interestingly, if I open the image with Squeak 4.2.1, and save it and then open it in Cog, Cog does not crash. So, there might be some inconsistencies in our image loading/saving routines.
Could someone give it a try on a VM instrumented for debugging and explain me what the problem is? There could be a bug in the RoarVM with filing out the image, but it is strange that Squeak 4.2.1 handles it just fine.
Thanks Stefan
[1] http://soft.vub.ac.be/~smarr/roarvm/squeak-saved-with-rvm.zip
On Mon, Jul 18, 2011 at 02:02:10PM +0200, Stefan Marr wrote:
Hi:
I recently added support to load images that have been saved with the CogVM in image format 6505 to the RoarVM. However, now I noticed that the CogVM/StackVM crash when they are asked to load an image which is saved by our RoarVM.
Interestingly, the Squeak 4.2.1 and earlier are able to load this image.
To reproduce the problem, I put online an image [1] which is a stock Squeak4.2-10966, just downloaded, opened and saved with the RoarVM.
The Cog/StackVM fail to open it with an error in "0 org.squeakfoundation.Squeak 0x00033462 readImageFromFileHeapSizeStartingAt + 3058"
Interestingly, if I open the image with Squeak 4.2.1, and save it and then open it in Cog, Cog does not crash. So, there might be some inconsistencies in our image loading/saving routines.
Could someone give it a try on a VM instrumented for debugging and explain me what the problem is? There could be a bug in the RoarVM with filing out the image, but it is strange that Squeak 4.2.1 handles it just fine.
Thanks Stefan
[1] http://soft.vub.ac.be/~smarr/roarvm/squeak-saved-with-rvm.zip
The difference between the 6505 format at 6504 format is the word reversal for floats when running on little-endian. That is the only difference in the image itself, but Cog/StackInterpreter also add some additional information in the image file header. This means that if you load a 6505 format image into RoarVM, and later save it in that same format (as opposed to converting back to 6504), then your image file header will be missing some information that an actual Cog VM would expect to see.
For the standard VM, I adopted the convention of accepting a 6505 format image, converting the float word ordering back to 6504 format at load time, and always saving it as 6504. Since Cog and StackVM know how to read a 6504 format, and the standard VM knows how to read a 6505 format, everything works as expected with this policy.
The ImageFormat package in SqueakSource/VMMaker is my attempt to document these image formats as they evolve.
Dave
Hi Dave:
On 18/07/11 14:34, David T. Lewis wrote:
On Mon, Jul 18, 2011 at 02:02:10PM +0200, Stefan Marr wrote:
However, now I noticed that the CogVM/StackVM crash when they are asked to load an image which is saved by our RoarVM.
The Cog/StackVM fail to open it with an error in "0 org.squeakfoundation.Squeak 0x00033462 readImageFromFileHeapSizeStartingAt + 3058"
[1] http://soft.vub.ac.be/~smarr/roarvm/squeak-saved-with-rvm.zip
The difference between the 6505 format at 6504 format is the word reversal for floats when running on little-endian. That is the only difference in the image itself, but Cog/StackInterpreter also add some additional information in the image file header. This means that if you load a 6505 format image into RoarVM, and later save it in that same format (as opposed to converting back to 6504), then your image file header will be missing some information that an actual Cog VM would expect to see.
I should have left out the mentioning of 6505, sorry for the confusion. I do not think that it has anything to do with that.
I took the standard stock Squeak 4.2 image, which is as far as I can tell published as 6504.
Now, I open and save it with the RoarVM, and then try to open it with Stack/CogVM, which crashes.
For the standard VM, I adopted the convention of accepting a 6505 format image, converting the float word ordering back to 6504 format at load time, and always saving it as 6504. Since Cog and StackVM know how to read a 6504 format, and the standard VM knows how to read a 6505 format, everything works as expected with this policy.
Just as a side note, since I think it is unrelated to the issue I am seeing: Yes, that is exactly, what I do to. I might have missed some bits, since the cs file I found unfortunately does not contain diffs. So I could not figure out exactly what changed, but well. At least I got the necessary functionality which allows me to open images saved with Cog. For reference: https://github.com/smarr/RoarVM/commit/c5652325710fd90c6acd1164713758b146af8... Crediting you ;)
Best regards Stefan
On Mon, Jul 18, 2011 at 03:01:07PM +0200, Stefan Marr wrote:
Hi Dave:
On 18/07/11 14:34, David T. Lewis wrote:
On Mon, Jul 18, 2011 at 02:02:10PM +0200, Stefan Marr wrote:
However, now I noticed that the CogVM/StackVM crash when they are asked to load an image which is saved by our RoarVM.
The Cog/StackVM fail to open it with an error in "0 org.squeakfoundation.Squeak 0x00033462 readImageFromFileHeapSizeStartingAt + 3058"
[1] http://soft.vub.ac.be/~smarr/roarvm/squeak-saved-with-rvm.zip
The difference between the 6505 format at 6504 format is the word reversal for floats when running on little-endian. That is the only difference in the image itself, but Cog/StackInterpreter also add some additional information in the image file header. This means that if you load a 6505 format image into RoarVM, and later save it in that same format (as opposed to converting back to 6504), then your image file header will be missing some information that an actual Cog VM would expect to see.
I should have left out the mentioning of 6505, sorry for the confusion. I do not think that it has anything to do with that.
I took the standard stock Squeak 4.2 image, which is as far as I can tell published as 6504.
Now, I open and save it with the RoarVM, and then try to open it with Stack/CogVM, which crashes.
Ah, I see. I suspect that one of the values that you write to the image header when saving from RoarVM may be causing the problem. It may be that the standard interpreter VM is more permissive than Cog with respect to loading the image, so perhaps you are saving a value (e.g. related to memory size) that causes a problem on Cog but not on the standard interpreter. I am away and cannot look at it right now, but reading your saved image with a StackInterpreterSimulator might point to the problem.
HTH, Dave
Hi:
On 18/07/11 15:01, Stefan Marr wrote:
Hi Dave:
On 18/07/11 14:34, David T. Lewis wrote:
On Mon, Jul 18, 2011 at 02:02:10PM +0200, Stefan Marr wrote:
However, now I noticed that the CogVM/StackVM crash when they are asked to load an image which is saved by our RoarVM.
The Cog/StackVM fail to open it with an error in "0 org.squeakfoundation.Squeak 0x00033462 readImageFromFileHeapSizeStartingAt + 3058"
[1] http://soft.vub.ac.be/~smarr/roarvm/squeak-saved-with-rvm.zip
I took the standard stock Squeak 4.2 image, which is as far as I can tell published as 6504.
Now, I open and save it with the RoarVM, and then try to open it with Stack/CogVM, which crashes.
I tried to dig into this a bit more.
The latest StackInterpreterSimulator in the image provided by Eliot on the Cog branch seems to read the image [1] just fine. It seems to hang, but might also be that I am to inpatient.
Anyway, a debug build of http://www.squeakvm.org/unix/release/Squeak-4.4.7.2357-src.tar.gz loads the image, too.
The available 5.x mac releases fail to load it: http://www.squeakvm.org/mac/
So, I took the Cog branch and the CoreVM.xcodeproj and got a VM compiled:
#0 0x00104f80 in initializeInterpreter (bytesToShift=339693560) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:17479 ccIndex = 0 cct = 394155372 classArrayClass = 401932980 classArrayObj = 400674088 header = 317718813 i = 17402624 i1 = 1 i11 = 411556616 i2 = 317431 i3 = -1073746952 i4 = 11 index = -2 oop = 10
#1 0x00123fc0 in readImageFromFileHeapSizeStartingAt (f=0xa064d8e0, desiredHeapSize=536870912, imageOffset=0) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:39218 #2 0x000652dc in main (argc=1, argv=0xbffff278, envp=0xbffff280) at /tmp/Cog/macbuild/../platforms/Mac OS/vm/sqMacMain.c:445
Where the code is:
17474 GIV(thisClassIndex) = i4; 17475 } 17476 } 17477 for (i4 = (InstanceSpecificationIndex + 1); i4 <= (lengthOf(classArrayObj)); i4 += 1) { 17478 oop = longAt((classArrayObj + BaseHeaderSize) + (i4 << ShiftForWord)); -> 17479 if ((((oop & 1) == 0) 17480 && (((((usqInt) (longAt(oop))) >> 8) & 15) >= 8)) 17481 && (((lengthOf(oop)) == 5) 17482 && ((strncmp("Array", firstFixedField(oop), 5)) == 0))) { 17483 GIV(classNameIndex) = i4; 17484 } 17485 }
Any guesses what is going on here?
Thanks Stefan
On 19 July 2011 02:43, Stefan Marr squeak@stefan-marr.de wrote:
Hi:
On 18/07/11 15:01, Stefan Marr wrote:
Hi Dave:
On 18/07/11 14:34, David T. Lewis wrote:
On Mon, Jul 18, 2011 at 02:02:10PM +0200, Stefan Marr wrote:
However, now I noticed that the CogVM/StackVM crash when they are asked to load an image which is saved by our RoarVM.
The Cog/StackVM fail to open it with an error in "0 org.squeakfoundation.Squeak 0x00033462 readImageFromFileHeapSizeStartingAt + 3058"
[1] http://soft.vub.ac.be/~smarr/roarvm/squeak-saved-with-rvm.zip
I took the standard stock Squeak 4.2 image, which is as far as I can tell published as 6504.
Now, I open and save it with the RoarVM, and then try to open it with Stack/CogVM, which crashes.
I tried to dig into this a bit more.
The latest StackInterpreterSimulator in the image provided by Eliot on the Cog branch seems to read the image [1] just fine. It seems to hang, but might also be that I am to inpatient.
Anyway, a debug build of http://www.squeakvm.org/unix/release/Squeak-4.4.7.2357-src.tar.gz loads the image, too.
The available 5.x mac releases fail to load it: http://www.squeakvm.org/mac/
So, I took the Cog branch and the CoreVM.xcodeproj and got a VM compiled:
#0 0x00104f80 in initializeInterpreter (bytesToShift=339693560) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:17479 ccIndex = 0 cct = 394155372 classArrayClass = 401932980 classArrayObj = 400674088 header = 317718813 i = 17402624 i1 = 1 i11 = 411556616 i2 = 317431 i3 = -1073746952 i4 = 11 index = -2 oop = 10
#1 0x00123fc0 in readImageFromFileHeapSizeStartingAt (f=0xa064d8e0, desiredHeapSize=536870912, imageOffset=0) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:39218 #2 0x000652dc in main (argc=1, argv=0xbffff278, envp=0xbffff280) at /tmp/Cog/macbuild/../platforms/Mac OS/vm/sqMacMain.c:445
Where the code is:
17474 GIV(thisClassIndex) = i4; 17475 } 17476 } 17477 for (i4 = (InstanceSpecificationIndex + 1); i4 <= (lengthOf(classArrayObj)); i4 += 1) { 17478 oop = longAt((classArrayObj + BaseHeaderSize) + (i4 << ShiftForWord)); -> 17479 if ((((oop & 1) == 0) 17480 && (((((usqInt) (longAt(oop))) >> 8) & 15) >= 8)) 17481 && (((lengthOf(oop)) == 5) 17482 && ((strncmp("Array", firstFixedField(oop), 5)) == 0))) { 17483 GIV(classNameIndex) = i4; 17484 } 17485 }
Any guesses what is going on here?
looks like it tries to discover the name ivar index for class objects.
Thanks Stefan
On Tue, Jul 19, 2011 at 01:43:33AM +0200, Stefan Marr wrote:
Hi:
On 18/07/11 15:01, Stefan Marr wrote:
Hi Dave:
On 18/07/11 14:34, David T. Lewis wrote:
On Mon, Jul 18, 2011 at 02:02:10PM +0200, Stefan Marr wrote:
However, now I noticed that the CogVM/StackVM crash when they are asked to load an image which is saved by our RoarVM.
The Cog/StackVM fail to open it with an error in "0 org.squeakfoundation.Squeak 0x00033462 readImageFromFileHeapSizeStartingAt + 3058"
[1] http://soft.vub.ac.be/~smarr/roarvm/squeak-saved-with-rvm.zip
I took the standard stock Squeak 4.2 image, which is as far as I can tell published as 6504.
Now, I open and save it with the RoarVM, and then try to open it with Stack/CogVM, which crashes.
I tried to dig into this a bit more.
The latest StackInterpreterSimulator in the image provided by Eliot on the Cog branch seems to read the image [1] just fine. It seems to hang, but might also be that I am to inpatient.
Anyway, a debug build of http://www.squeakvm.org/unix/release/Squeak-4.4.7.2357-src.tar.gz loads the image, too.
The available 5.x mac releases fail to load it: http://www.squeakvm.org/mac/
So, I took the Cog branch and the CoreVM.xcodeproj and got a VM compiled:
#0 0x00104f80 in initializeInterpreter (bytesToShift=339693560) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:17479 ccIndex = 0 cct = 394155372 classArrayClass = 401932980 classArrayObj = 400674088 header = 317718813 i = 17402624 i1 = 1 i11 = 411556616 i2 = 317431 i3 = -1073746952 i4 = 11 index = -2 oop = 10
#1 0x00123fc0 in readImageFromFileHeapSizeStartingAt (f=0xa064d8e0, desiredHeapSize=536870912, imageOffset=0) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:39218 #2 0x000652dc in main (argc=1, argv=0xbffff278, envp=0xbffff280) at /tmp/Cog/macbuild/../platforms/Mac OS/vm/sqMacMain.c:445
Where the code is:
17474 GIV(thisClassIndex) = i4; 17475 } 17476 } 17477 for (i4 = (InstanceSpecificationIndex + 1); i4 <= (lengthOf(classArrayObj)); i4 += 1) { 17478 oop = longAt((classArrayObj + BaseHeaderSize) + (i4 << ShiftForWord)); -> 17479 if ((((oop & 1) == 0) 17480 && (((((usqInt) (longAt(oop))) >> 8) & 15) >= 8)) 17481 && (((lengthOf(oop)) == 5) 17482 && ((strncmp("Array", firstFixedField(oop), 5)) == 0))) { 17483 GIV(classNameIndex) = i4; 17484 } 17485 }
Any guesses what is going on here?
The following looks suspiciously like an arithmetic overflow on a variable declared as sqInt:
i3 = -1073746952
This is in the initializeInterpreter function, and i3 would probably be a local var or parameter from some method that got inlined into that method. If in fact it is an overflow, that might be a symptom of e.g. some uninitialized variable that allows a scan through object memory to run amok.
This is quite possibly a red herring, inasmuch as the i3 variable does not appear to have anything to do with the line of code that crashed, but it does look odd, so you might want to look at the initializeInterpreter() function in src/vm/gcc3x-cointerp.c and see where it is coming from.
I don't know if I'm looking at the same version of the oscog VMMaker that you are using, but the i3 variable that I get is here:
/* begin flushMethodCache */ for (i3 = 1; i3 <= MethodCacheSize; i3 += 1) { GIV(methodCache)[i3] = 0; }
...which looks perfectly innocent to me, so again, maybe just a red herring. But any time I see a negative sqInt in the context of scanning object memory, it catches my attention.
Dave
Hi:
Just for completeness:
/tmp/Cog$ svn info Path: . URL: http://squeakvm.org/svn/squeak/branches/Cog Repository Root: http://squeakvm.org/svn/squeak Repository UUID: fa1542d4-bde8-0310-ad64-8ed1123d492a Revision: 2462 Node Kind: directory Schedule: normal Last Changed Author: eliot Last Changed Rev: 2462 Last Changed Date: 2011-07-16 00:35:48 +0200 (Sat, 16 Jul 2011)
On 19/07/11 04:07, David T. Lewis wrote:
On Tue, Jul 19, 2011 at 01:43:33AM +0200, Stefan Marr wrote:
So, I took the Cog branch and the CoreVM.xcodeproj and got a VM compiled:
#0 0x00104f80 in initializeInterpreter (bytesToShift=339693560) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:17479 ccIndex = 0 cct = 394155372 classArrayClass = 401932980 classArrayObj = 400674088 header = 317718813 i = 17402624 i1 = 1 i11 = 411556616 i2 = 317431 i3 = -1073746952 i4 = 11 index = -2 oop = 10
#1 0x00123fc0 in readImageFromFileHeapSizeStartingAt (f=0xa064d8e0, desiredHeapSize=536870912, imageOffset=0) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:39218 #2 0x000652dc in main (argc=1, argv=0xbffff278, envp=0xbffff280) at /tmp/Cog/macbuild/../platforms/Mac OS/vm/sqMacMain.c:445
Where the code is:
17474 GIV(thisClassIndex) = i4; 17475 } 17476 } 17477 for (i4 = (InstanceSpecificationIndex + 1); i4<= (lengthOf(classArrayObj)); i4 += 1) { 17478 oop = longAt((classArrayObj + BaseHeaderSize) + (i4<< ShiftForWord)); -> 17479 if ((((oop& 1) == 0) 17480&& (((((usqInt) (longAt(oop)))>> 8)& 15)>= 8)) 17481&& (((lengthOf(oop)) == 5) 17482&& ((strncmp("Array", firstFixedField(oop), 5)) == 0))) { 17483 GIV(classNameIndex) = i4; 17484 } 17485 }
Any guesses what is going on here? The following looks suspiciously like an arithmetic overflow on a variable declared as sqInt:
i3 = -1073746952
Well, that is just thanks to the generated code which does the initializations on the spot. And, we just did not yet get to the code actually using i3.
Interestingly, those initialization loops do more work then strictly necessary, which triggers the 'bug' here. Well, even if there is some problem in the image we generate, Cog just should not crash at that point.
So, to avoid the additional work, i.e., unnecessary iterations over the rest of the object/array, I added the following breaks:
@@ -17472,6 +17472,7 @@ for (i4 = (InstanceSpecificationIndex + 1); i4 <= (lengthOf(classArrayClass)); i4 += 1) { if ((longAt((classArrayClass + BaseHeaderSize) + (i4 << ShiftForWord))) == classArrayObj) { GIV(thisClassIndex) = i4; + break; } } for (i4 = (InstanceSpecificationIndex + 1); i4 <= (lengthOf(classArrayObj)); i4 += 1) { @@ -17481,6 +17482,7 @@ && (((lengthOf(oop)) == 5) && ((strncmp("Array", firstFixedField(oop), 5)) == 0))) { GIV(classNameIndex) = i4; + break; } }
This results then in a VM which actually comes up and is functioning as far as I can tell. However, I get the following output, which might be an indication what is wrong with my image:
mprotect(x,y,PROT_READ | PROT_WRITE): Cannot allocate memory squeak: could not load plugin `MiscPrimitivePlugin'
(objectAfter(falseObject())) == (trueObject()) 9202
(objectAfter(falseObject())) == (trueObject()) 9202
...
--> "(objectAfter(falseObject())) == (trueObject()) 9202" got repeated 1985 times in total.
After all that, and after rewriting the generated code of the relevant loop into something readable, I think I understand what is going on:
Looks like I have the object representing the Array class at hand, and the initialization tries to identify which index into a class object gives you the actual class name: it sets GIV(classNameIndex) (GIV = global interpreter variable, I suppose). And well, after finding this, as I said it actually is continuing to loop over the object slots instead of stopping which results in hitting index 11 into that particular object.
I am not sure whether I understand it correctly, but from my understanding it is accessing the following instance variables by index:
1: methodDict: 2: format: 6402 3: instanceVariables: nil 4: organization: 5: subclasses: {WeakArray . ActionSequence . WeakActionSequence . Cubic} 6: name: #Array 7: classPool: a Dictionary() 8: sharedPools: nil 9: environment: nil 10: category: #''Collections-Arrayed''
And the accessing 11 fails because it is not actually an instance variable.
Is my understanding correct so far?
If so, I guess the lengthOf(classArrayObj) goes wrong with my image, which seems to be strange somehow, because I would guess such a bug would trigger elsewhere, too.
Best regards Stefan
Dave
FYI, and consistent with what you describe below, I tried running your Squeak4.2-10966-rvm-saved.image in a StackInterpreterSimulator. I then resaved the image from a standard interpreter VM, and ran that re-saved image in a StackInterpreterSimulator. The one that was re-saved can be successfully loaded into the simulator, while the RoarVM saved image fails in #initializeExtraClassInstVarIndices, called from #initializeInterpreter:
So the issue does appear to be in the image itself as opposed to the image header (as I had earlier speculated).
The failure occurs in the same general area as you describe, but the simulator detects the problem sooner (it catches an "unaligned access" at index 7, never gets to index 11). That's about all I can say about it, but you might want to look at it from the simulator and see if it gives you some better clues.
To reproduce what I did, use an image with Eliot's latest oscog VMMaker, then evaluate:
StackInterpreterSimulator new openOn: 'Squeak4.2-10966-rvm-saved.image'
This will load your object memory and catch the failure in a debugger at #initializeExtraClassVarIndices.
I note also that the standard interpreter does not have the #initializeExtraClassVarIndices method at all, so if the failure is specific to that initialization (as opposed to some general corruption of the object memory), that would help explain why the standard VM can load the image.
Dave
On Tue, Jul 19, 2011 at 11:38:54AM +0200, Stefan Marr wrote:
Hi:
Just for completeness:
/tmp/Cog$ svn info Path: . URL: http://squeakvm.org/svn/squeak/branches/Cog Repository Root: http://squeakvm.org/svn/squeak Repository UUID: fa1542d4-bde8-0310-ad64-8ed1123d492a Revision: 2462 Node Kind: directory Schedule: normal Last Changed Author: eliot Last Changed Rev: 2462 Last Changed Date: 2011-07-16 00:35:48 +0200 (Sat, 16 Jul 2011)
On 19/07/11 04:07, David T. Lewis wrote:
On Tue, Jul 19, 2011 at 01:43:33AM +0200, Stefan Marr wrote:
So, I took the Cog branch and the CoreVM.xcodeproj and got a VM compiled:
#0 0x00104f80 in initializeInterpreter (bytesToShift=339693560) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:17479 ccIndex = 0 cct = 394155372 classArrayClass = 401932980 classArrayObj = 400674088 header = 317718813 i = 17402624 i1 = 1 i11 = 411556616 i2 = 317431 i3 = -1073746952 i4 = 11 index = -2 oop = 10
#1 0x00123fc0 in readImageFromFileHeapSizeStartingAt (f=0xa064d8e0, desiredHeapSize=536870912, imageOffset=0) at /tmp/Cog/macbuild/../src/vm/gcc3x-cointerp.c:39218 #2 0x000652dc in main (argc=1, argv=0xbffff278, envp=0xbffff280) at /tmp/Cog/macbuild/../platforms/Mac OS/vm/sqMacMain.c:445
Where the code is:
17474 GIV(thisClassIndex) = i4; 17475 } 17476 } 17477 for (i4 = (InstanceSpecificationIndex + 1); i4<= (lengthOf(classArrayObj)); i4 += 1) { 17478 oop = longAt((classArrayObj + BaseHeaderSize) + (i4<< ShiftForWord)); -> 17479 if ((((oop& 1) == 0) 17480&& (((((usqInt) (longAt(oop)))>> 8)& 15)>= 8)) 17481&& (((lengthOf(oop)) == 5) 17482&& ((strncmp("Array", firstFixedField(oop), 5)) == 0))) { 17483 GIV(classNameIndex) = i4; 17484 } 17485 }
Any guesses what is going on here? The following looks suspiciously like an arithmetic overflow on a variable declared as sqInt:
i3 = -1073746952Well, that is just thanks to the generated code which does the initializations on the spot. And, we just did not yet get to the code actually using i3.
Interestingly, those initialization loops do more work then strictly necessary, which triggers the 'bug' here. Well, even if there is some problem in the image we generate, Cog just should not crash at that point.
So, to avoid the additional work, i.e., unnecessary iterations over the rest of the object/array, I added the following breaks:
@@ -17472,6 +17472,7 @@ for (i4 = (InstanceSpecificationIndex + 1); i4 <= (lengthOf(classArrayClass)); i4 += 1) { if ((longAt((classArrayClass + BaseHeaderSize) + (i4 << ShiftForWord))) == classArrayObj) { GIV(thisClassIndex) = i4;
break; }(lengthOf(classArrayObj)); i4 += 1) { @@ -17481,6 +17482,7 @@ && (((lengthOf(oop)) == 5) && ((strncmp("Array", firstFixedField(oop), 5)) == 0))) { GIV(classNameIndex) = i4;
break; }This results then in a VM which actually comes up and is functioning as far as I can tell. However, I get the following output, which might be an indication what is wrong with my image:
mprotect(x,y,PROT_READ | PROT_WRITE): Cannot allocate memory squeak: could not load plugin `MiscPrimitivePlugin'
(objectAfter(falseObject())) == (trueObject()) 9202
(objectAfter(falseObject())) == (trueObject()) 9202
...
--> "(objectAfter(falseObject())) == (trueObject()) 9202" got repeated 1985 times in total.
After all that, and after rewriting the generated code of the relevant loop into something readable, I think I understand what is going on:
Looks like I have the object representing the Array class at hand, and the initialization tries to identify which index into a class object gives you the actual class name: it sets GIV(classNameIndex) (GIV = global interpreter variable, I suppose). And well, after finding this, as I said it actually is continuing to loop over the object slots instead of stopping which results in hitting index 11 into that particular object.
I am not sure whether I understand it correctly, but from my understanding it is accessing the following instance variables by index:
1: methodDict: 2: format: 6402 3: instanceVariables: nil 4: organization: 5: subclasses: {WeakArray . ActionSequence . WeakActionSequence . Cubic} 6: name: #Array 7: classPool: a Dictionary() 8: sharedPools: nil 9: environment: nil 10: category: #''Collections-Arrayed''
And the accessing 11 fails because it is not actually an instance variable.
Is my understanding correct so far?
If so, I guess the lengthOf(classArrayObj) goes wrong with my image, which seems to be strange somehow, because I would guess such a bug would trigger elsewhere, too.
Best regards Stefan
Dave
Hi Stefan,
On Tue, Jul 19, 2011 at 2:38 AM, Stefan Marr squeak@stefan-marr.de wrote:
Hi:
Just for completeness:
/tmp/Cog$ svn info Path: . URL: http://squeakvm.org/svn/**squeak/branches/Coghttp://squeakvm.org/svn/squeak/branches/Cog Repository Root: http://squeakvm.org/svn/squeak Repository UUID: fa1542d4-bde8-0310-ad64-**8ed1123d492a Revision: 2462 Node Kind: directory Schedule: normal Last Changed Author: eliot Last Changed Rev: 2462 Last Changed Date: 2011-07-16 00:35:48 +0200 (Sat, 16 Jul 2011)
On 19/07/11 04:07, David T. Lewis wrote:
On Tue, Jul 19, 2011 at 01:43:33AM +0200, Stefan Marr wrote:
So, I took the Cog branch and the CoreVM.xcodeproj and got a VM compiled:
#0 0x00104f80 in initializeInterpreter (bytesToShift=339693560) at /tmp/Cog/macbuild/../src/vm/**gcc3x-cointerp.c:17479 ccIndex = 0 cct = 394155372 classArrayClass = 401932980 classArrayObj = 400674088 header = 317718813 i = 17402624 i1 = 1 i11 = 411556616 i2 = 317431 i3 = -1073746952 i4 = 11 index = -2 oop = 10
#1 0x00123fc0 in readImageFromFileHeapSizeStart**ingAt (f=0xa064d8e0, desiredHeapSize=536870912, imageOffset=0) at /tmp/Cog/macbuild/../src/vm/**gcc3x-cointerp.c:39218 #2 0x000652dc in main (argc=1, argv=0xbffff278, envp=0xbffff280) at /tmp/Cog/macbuild/../**platforms/Mac OS/vm/sqMacMain.c:445
Where the code is:
17474 GIV(thisClassIndex) = i4; 17475 } 17476 } 17477 for (i4 = (InstanceSpecificationIndex + 1); i4<= (lengthOf(classArrayObj)); i4 += 1) { 17478 oop = longAt((classArrayObj + BaseHeaderSize) + (i4<< ShiftForWord)); -> 17479 if ((((oop& 1) == 0) 17480&& (((((usqInt) (longAt(oop)))>> 8)& 15)>= 8)) 17481&& (((lengthOf(oop)) == 5) 17482&& ((strncmp("Array", firstFixedField(oop), 5)) == 0))) { 17483 GIV(classNameIndex) = i4; 17484 } 17485 }
Any guesses what is going on here? The following looks suspiciously like an arithmetic overflow on a variable declared as sqInt:
i3 = -1073746952Well, that is just thanks to the generated code which does the initializations on the spot. And, we just did not yet get to the code actually using i3.
Interestingly, those initialization loops do more work then strictly necessary, which triggers the 'bug' here. Well, even if there is some problem in the image we generate, Cog just should not crash at that point.
So, to avoid the additional work, i.e., unnecessary iterations over the rest of the object/array, I added the following breaks:
@@ -17472,6 +17472,7 @@ for (i4 = (InstanceSpecificationIndex + 1); i4 <= (lengthOf(classArrayClass)); i4 += 1) { if ((longAt((classArrayClass + BaseHeaderSize) + (i4 << ShiftForWord))) == classArrayObj) { GIV(thisClassIndex) = i4;
break; }} for (i4 = (InstanceSpecificationIndex + 1); i4 <=
(lengthOf(classArrayObj)); i4 += 1) { @@ -17481,6 +17482,7 @@ && (((lengthOf(oop)) == 5)
&& ((strncmp("Array", firstFixedField(oop), 5)) == 0))) { GIV(classNameIndex) = i4;
break; }This results then in a VM which actually comes up and is functioning as far as I can tell. However, I get the following output, which might be an indication what is wrong with my image:
mprotect(x,y,PROT_READ | PROT_WRITE): Cannot allocate memory squeak: could not load plugin `MiscPrimitivePlugin'
(objectAfter(falseObject())) == (trueObject()) 9202
(objectAfter(falseObject())) == (trueObject()) 9202
...
--> "(objectAfter(falseObject())) == (trueObject()) 9202" got repeated 1985 times in total.
After all that, and after rewriting the generated code of the relevant loop into something readable, I think I understand what is going on:
Looks like I have the object representing the Array class at hand, and the initialization tries to identify which index into a class object gives you the actual class name: it sets GIV(classNameIndex) (GIV = global interpreter variable, I suppose). And well, after finding this, as I said it actually is continuing to loop over the object slots instead of stopping which results in hitting index 11 into that particular object.
I am not sure whether I understand it correctly, but from my understanding it is accessing the following instance variables by index:
1: methodDict: 2: format: 6402 3: instanceVariables: nil 4: organization: 5: subclasses: {WeakArray . ActionSequence . WeakActionSequence . Cubic} 6: name: #Array 7: classPool: a Dictionary() 8: sharedPools: nil 9: environment: nil 10: category: #''Collections-Arrayed''
And the accessing 11 fails because it is not actually an instance variable.
Doh! It's a one-relative/zero-relative thing. lengthOf: returns the number of fields in a pointer object, but i is a zero-relative variable, so the loops need to read
InstanceSpecificationIndex + 1 to: (objectMemory lengthOf: classArrayClass) - 1 do: [:i| (objectMemory fetchPointer: i ofObject: classArrayClass) = classArrayObj ifTrue: [thisClassIndex := i]].
Thanks for finding this!
Is my understanding correct so far?
If so, I guess the lengthOf(classArrayObj) goes wrong with my image, which seems to be strange somehow, because I would guess such a bug would trigger elsewhere, too.
Best regards Stefan
Dave
Hi Eliot:
On 19/07/11 19:38, Eliot Miranda wrote:
Doh! It's a one-relative/zero-relative thing. lengthOf: returns the number of fields in a pointer object, but i is a zero-relative variable, so the loops need to read
InstanceSpecificationIndex + 1 to: (objectMemory lengthOf: classArrayClass) - 1 do: [:i| (objectMemory fetchPointer: i ofObject: classArrayClass) = classArrayObj ifTrue: [thisClassIndex := i]].
Thanks for finding this!
Thanks for fixing it! :)
Just tested you latest binary, and it works. Thanks
Stefan
vm-dev@lists.squeakfoundation.org
-
David T. Lewis -
Eliot Miranda -
Igor Stasenko -
Stefan Marr