[squeak-dev] Re: [Vm-dev] [Pharo-dev] strange array instance

Stéphane Ducasse stephane.ducasse at inria.fr
Wed Mar 9 11:08:27 UTC 2016


Hi eliot 

Do not worry.
Now would it be possible to have a test or something like that that would help us
to 
	- document this part of the system
	- check that the system is in a coherent state

Stef


> On 09 Mar 2016, at 10:26, Eliot Miranda <eliot.miranda at gmail.com> wrote:
> 
> Um, under /no/ circumstances do what I'm suggesting below.  It will screw up 320bit images completely.  I've just turned odd SmallIntegers into SmallFloat54's in 32-bit Squeak.  I hope I won't destroy too many people's images.  This is perhaps my worst blunder.
> 
> On Tue, Mar 8, 2016 at 10:38 AM, Eliot Miranda <eliot.miranda at gmail.com <mailto:eliot.miranda at gmail.com>> wrote:
> Hi Pavel,
> 
> On Tue, Mar 8, 2016 at 1:40 AM, Pavel Krivanek <pavel.krivanek at gmail.com <mailto:pavel.krivanek at gmail.com>> wrote:
> Hi, 
> 
> there is a strange array of size 1024 in the Pharo image that is not eaten by garbage collector even if no object points to it. I guess it is related to Spur.
> 
> ((Array allInstances select: [ :i | i size = 1024 ])
>  	select: [ :a | a second = SmallInteger  ]) pointersTo
> 
> What is it and is it accessible somehow?
> 
> It is indeed the first class table page.  It looks like an old bug.  The oldest Spur image I can start up has the problem.   The way that class table pages are hidden is that they have a class index which is not that of Array.  Array is at index 52 in the class table (51 zero-relative, Array identityHash = 51), but it is also at index 17 (16 zero-relative).  Array allInstances looks for objects whose class index is 51, hence not seeing the class table pages, which all (but the first) have class index 16.
> 
> Luckily we can fix the problem easily, and we can fix another problem at the same time, which is that SmallFloat64 has the wrong hash.
> 
> The class table is two-level; a (hidden) array of Arrays.  Classes are in the class table at their identityHashes (so that to instantiate a class one copies the class's identity hash into the classIndex of the new instance instead of having to search the class table).  But when SmallFloat64 was created in the 32-bit image I forgot to give it the right hash and store it in the class table.  (there are no SmallFloat64 instances in a 32-bit image, only in 64-bits).
> 
> So we can first change SmallFloat64's hash to 3, which is what the Spur 64-bit VM and image require, and enter it into the first class table page, and then we can make the first class table page disappear.
> 
> 
> | firstClassTablePage clone |
> "The first class table page is the first Array instance. Of course this is a bug."
> firstClassTablePage := Array someInstance.
> 
> "It should contain only nil and classes."
> self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e isBehavior]]).
> 
> "And its first 17 elements should only be the immediate classes and Array, since Array is used as the class pun for class table pages, with index 17 (16 zero-relative)"
> self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet.
> 
> "It just so happens that the second Array is the specialSelectors"
> self assert: Array someInstance nextInstance == Smalltalk specialSelectors.
> 
> "Store SmallFloat64 in the first class table page at index 4 (3 zero-relative)."
> firstClassTablePage at: 4 put: SmallFloat64.
> 
> "Use the secret set identity hash primitive to set SmallFloat64's identityHash to 3."
> SmallFloat64 tryPrimitive: 161 withArgs: #(3).
> 
> "Now create an object that looks like Array class, but has identityHash 16."
> "Take a shallow copy of Array class."
> clone := Array shallowCopy.
> "Change it into an Array."
> Array adoptInstance: clone.
> "Set its identityHash to 16."
> clone tryPrimitive: 161 withArgs: #(16).
> "Use the adoptInstance: primitive to ``set the class of the firstClassTablePage to the cone''.
>  or, more accurately, to set the firstClassTablePage's class index to 16."
> clone tryPrimitive: 160 withArgs: {firstClassTablePage}
> 
> 
> With the above done, we can check that everything is ok."
> self assert: SmallFloat64 identityHash = 3.
> self assert: Array someInstance == Smalltalk specialSelectors.
> "A class table page is 1024 slots, contains only nil or behaviours, and contains at least one behaviour (is not all nils).  There shouldn't be any that we can find."
> self assert: (self systemNavigation allObjects select: [:o| o isArray and: [o size = 1024 and: [(o allSatisfy: [:e| e == nil or: [e isBehavior]]) and: [o anySatisfy: [:e| e isBehavior]]]]]) size = 0
> 
> 
> I recommend you create an update map.  Then create a version of kernel with a post-load action, written something like this:
> 
> 
> (Array someInstance size = 1014
>  and: [(Array someInstance allSatisfy: [:e| e == nil or: [e isBehavior]])
>  and: [(firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet]]) ifTrue:
> 	[| firstClassTablePage clone |
> 	firstClassTablePage := Array someInstance.
> 	self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e isBehavior]]).
> 	self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet.
> 	firstClassTablePage at: 4 put: SmallFloat64.
> 	SmallFloat64 tryPrimitive: 161 withArgs: #(3).
> 	clone := Array shallowCopy.
> 	Array adoptInstance: clone.
> 	clone tryPrimitive: 161 withArgs: #(16).
> 	clone tryPrimitive: 160 withArgs: {Array someInstance}
> 	self assert: SmallFloat64 identityHash = 3.
> 	self assert: (Array someInstance first: 4) = {nil. SmallInteger. Character. SmallFloat64}]
> 
> Then create a second update map to ensure that that version of Kernel is loaded.
> 
> I will do this for Squeak immediately.
> 
> 
> Apologies.
> 
> P.S. Interestingly enough, it shows what a horrible security hole the debugger primitives tryPrimitive:withArgs: and tryNamedPrimitive:withArgs: are.
> 
> Cheers,
> -- Pavel
> 
> _,,,^..^,,,_
> best, Eliot
> 
> 
> 
> -- 
> _,,,^..^,,,_
> best, Eliot

--------------------------------------------
Stéphane Ducasse
http://stephane.ducasse.free.fr
http://www.synectique.eu / http://www.pharo.org 
03 59 35 87 52
Assistant: Julie Jonas 
03 59 57 78 50
03 59 35 86 16

S. Ducasse - Inria
40, avenue Halley, 
Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
Villeneuve d'Ascq 59650
France

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20160309/7e83d3bf/attachment.htm


More information about the Squeak-dev mailing list