[squeak-dev] Re: [Vm-dev] [Pharo-dev] strange array instance

Wed Mar 9 16:58:30 UTC 2016

On Wed, Mar 9, 2016 at 3:08 AM, Stéphane Ducasse <stephane.ducasse at inria.fr>
wrote:

> Hi eliot
>
> Do not worry.
> Now would it be possible to have a test or something like that that would
> help us
> to
> - document this part of the system
> - check that the system is in a coherent state
>
> Stef
>

Yes, watch this space.  I should have the work done by tomorrow.  And the
64-bit system will be a tiny bit smaller and faster :-)

On 09 Mar 2016, at 10:26, Eliot Miranda <eliot.miranda at gmail.com> wrote:
>
> Um, under /no/ circumstances do what I'm suggesting below.  It will screw
> up 320bit images completely.  I've just turned odd SmallIntegers into
> SmallFloat54's in 32-bit Squeak.  I hope I won't destroy too many people's
> images.  This is perhaps my worst blunder.
>
> On Tue, Mar 8, 2016 at 10:38 AM, Eliot Miranda <eliot.miranda at gmail.com>
> wrote:
>
>> Hi Pavel,
>>
>> On Tue, Mar 8, 2016 at 1:40 AM, Pavel Krivanek <pavel.krivanek at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> there is a strange array of size 1024 in the Pharo image that is not
>>> eaten by garbage collector even if no object points to it. I guess it is
>>> related to Spur.
>>>
>>> ((Array allInstances select: [ :i | i size = 1024 ])
>>>   select: [ :a | a second = SmallInteger  ]) pointersTo
>>>
>>> What is it and is it accessible somehow?
>>>
>>
>> It is indeed the first class table page.  It looks like an old bug.  The
>> oldest Spur image I can start up has the problem.   The way that class
>> table pages are hidden is that they have a class index which is not that of
>> Array.  Array is at index 52 in the class table (51 zero-relative, Array
>> identityHash = 51), but it is also at index 17 (16 zero-relative).  Array
>> allInstances looks for objects whose class index is 51, hence not seeing
>> the class table pages, which all (but the first) have class index 16.
>>
>> Luckily we can fix the problem easily, and we can fix another problem at
>> the same time, which is that SmallFloat64 has the wrong hash.
>>
>> The class table is two-level; a (hidden) array of Arrays.  Classes are in
>> the class table at their identityHashes (so that to instantiate a class one
>> copies the class's identity hash into the classIndex of the new instance
>> instead of having to search the class table).  But when SmallFloat64 was
>> created in the 32-bit image I forgot to give it the right hash and store it
>> in the class table.  (there are no SmallFloat64 instances in a 32-bit
>> image, only in 64-bits).
>>
>> So we can first change SmallFloat64's hash to 3, which is what the Spur
>> 64-bit VM and image require, and enter it into the first class table page,
>> and then we can make the first class table page disappear.
>>
>>
>> | firstClassTablePage clone |
>> "The first class table page is the first Array instance. Of course this
>> is a bug."
>> firstClassTablePage := Array someInstance.
>>
>> "It should contain only nil and classes."
>> self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e
>> isBehavior]]).
>>
>> "And its first 17 elements should only be the immediate classes and
>> Array, since Array is used as the class pun for class table pages, with
>> index 17 (16 zero-relative)"
>> self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger.
>> Character. Array} asSet.
>>
>> "It just so happens that the second Array is the specialSelectors"
>> self assert: Array someInstance nextInstance == Smalltalk
>> specialSelectors.
>>
>> "Store SmallFloat64 in the first class table page at index 4 (3
>> zero-relative)."
>> firstClassTablePage at: 4 put: SmallFloat64.
>>
>> "Use the secret set identity hash primitive to set SmallFloat64's
>> identityHash to 3."
>> SmallFloat64 tryPrimitive: 161 withArgs: #(3).
>>
>> "Now create an object that looks like Array class, but has identityHash
>> 16."
>> "Take a shallow copy of Array class."
>> clone := Array shallowCopy.
>> "Change it into an Array."
>> Array adoptInstance: clone.
>> "Set its identityHash to 16."
>> clone tryPrimitive: 161 withArgs: #(16).
>> "Use the adoptInstance: primitive to ``set the class of the
>> firstClassTablePage to the cone''.
>>  or, more accurately, to set the firstClassTablePage's class index to 16."
>> clone tryPrimitive: 160 withArgs: {firstClassTablePage}
>>
>>
>> With the above done, we can check that everything is ok."
>> self assert: SmallFloat64 identityHash = 3.
>> self assert: Array someInstance == Smalltalk specialSelectors.
>> "A class table page is 1024 slots, contains only nil or behaviours, and
>> contains at least one behaviour (is not all nils).  There shouldn't be any
>> that we can find."
>> self assert: (self systemNavigation allObjects select: [:o| o isArray
>> and: [o size = 1024 and: [(o allSatisfy: [:e| e == nil or: [e isBehavior]])
>> and: [o anySatisfy: [:e| e isBehavior]]]]]) size = 0
>>
>>
>> I recommend you create an update map.  Then create a version of kernel
>> with a post-load action, written something like this:
>>
>>
>> (Array someInstance size = 1014
>>  and: [(Array someInstance allSatisfy: [:e| e == nil or: [e isBehavior]])
>>  and: [(firstClassTablePage first: 17) asSet = {nil. SmallInteger.
>> Character. Array} asSet]]) ifTrue:
>> [| firstClassTablePage clone |
>> firstClassTablePage := Array someInstance.
>> self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e
>> isBehavior]]).
>> self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger.
>> Character. Array} asSet.
>> firstClassTablePage at: 4 put: SmallFloat64.
>> SmallFloat64 tryPrimitive: 161 withArgs: #(3).
>> clone := Array shallowCopy.
>> Array adoptInstance: clone.
>> clone tryPrimitive: 161 withArgs: #(16).
>> clone tryPrimitive: 160 withArgs: {Array someInstance}
>> self assert: SmallFloat64 identityHash = 3.
>> self assert: (Array someInstance first: 4) = {nil. SmallInteger.
>> Character. SmallFloat64}]
>>
>> Then create a second update map to ensure that that version of Kernel is
>> loaded.
>>
>> I will do this for Squeak immediately.
>>
>>
>> Apologies.
>>
>> P.S. Interestingly enough, it shows what a horrible security hole the
>> debugger primitives tryPrimitive:withArgs: and tryNamedPrimitive:withArgs:
>> are.
>>
>> Cheers,
>>> -- Pavel
>>>
>>
>> _,,,^..^,,,_
>> best, Eliot
>>
>
>
>
> --
> _,,,^..^,,,_
> best, Eliot
>
>
> --------------------------------------------
> Stéphane Ducasse
> http://stephane.ducasse.free.fr
> http://www.synectique.eu / http://www.pharo.org
> 03 59 35 87 52
> Assistant: Julie Jonas
> 03 59 57 78 50
> 03 59 35 86 16
>
> S. Ducasse - Inria
> 40, avenue Halley,
> Parc Scientifique de la Haute Borne, Bât.A, Park Plaza
> Villeneuve d'Ascq 59650
> France
>
>
>
>
>

-- 
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20160309/b2423b9a/attachment-0001.htm