Hi Pavel,

On Tue, Mar 8, 2016 at 1:40 AM, Pavel Krivanek <pavel.krivanek at gmail.com>
wrote:

> Hi,
>
> there is a strange array of size 1024 in the Pharo image that is not eaten
> by garbage collector even if no object points to it. I guess it is related
> to Spur.
>
> ((Array allInstances select: [ :i | i size = 1024 ])
>   select: [ :a | a second = SmallInteger  ]) pointersTo
>
> What is it and is it accessible somehow?
>

It is indeed the first class table page.  It looks like an old bug.  The
oldest Spur image I can start up has the problem.   The way that class
table pages are hidden is that they have a class index which is not that of
Array.  Array is at index 52 in the class table (51 zero-relative, Array
identityHash = 51), but it is also at index 17 (16 zero-relative).  Array
allInstances looks for objects whose class index is 51, hence not seeing
the class table pages, which all (but the first) have class index 16.

Luckily we can fix the problem easily, and we can fix another problem at
the same time, which is that SmallFloat64 has the wrong hash.

The class table is two-level; a (hidden) array of Arrays.  Classes are in
the class table at their identityHashes (so that to instantiate a class one
copies the class's identity hash into the classIndex of the new instance
instead of having to search the class table).  But when SmallFloat64 was
created in the 32-bit image I forgot to give it the right hash and store it
in the class table.  (there are no SmallFloat64 instances in a 32-bit
image, only in 64-bits).

So we can first change SmallFloat64's hash to 3, which is what the Spur
64-bit VM and image require, and enter it into the first class table page,
and then we can make the first class table page disappear.


| firstClassTablePage clone |
"The first class table page is the first Array instance. Of course this is
a bug."
firstClassTablePage := Array someInstance.

"It should contain only nil and classes."
self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e
isBehavior]]).

"And its first 17 elements should only be the immediate classes and Array,
since Array is used as the class pun for class table pages, with index 17
(16 zero-relative)"
self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger.
Character. Array} asSet.

"It just so happens that the second Array is the specialSelectors"
self assert: Array someInstance nextInstance == Smalltalk specialSelectors.

"Store SmallFloat64 in the first class table page at index 4 (3
zero-relative)."
firstClassTablePage at: 4 put: SmallFloat64.

"Use the secret set identity hash primitive to set SmallFloat64's
identityHash to 3."
SmallFloat64 tryPrimitive: 161 withArgs: #(3).

"Now create an object that looks like Array class, but has identityHash 16."
"Take a shallow copy of Array class."
clone := Array shallowCopy.
"Change it into an Array."
Array adoptInstance: clone.
"Set its identityHash to 16."
clone tryPrimitive: 161 withArgs: #(16).
"Use the adoptInstance: primitive to ``set the class of the
firstClassTablePage to the cone''.
 or, more accurately, to set the firstClassTablePage's class index to 16."
clone tryPrimitive: 160 withArgs: {firstClassTablePage}


With the above done, we can check that everything is ok."
self assert: SmallFloat64 identityHash = 3.
self assert: Array someInstance == Smalltalk specialSelectors.
"A class table page is 1024 slots, contains only nil or behaviours, and
contains at least one behaviour (is not all nils).  There shouldn't be any
that we can find."
self assert: (self systemNavigation allObjects select: [:o| o isArray and:
[o size = 1024 and: [(o allSatisfy: [:e| e == nil or: [e isBehavior]]) and:
[o anySatisfy: [:e| e isBehavior]]]]]) size = 0


I recommend you create an update map.  Then create a version of kernel with
a post-load action, written something like this:


(Array someInstance size = 1014
 and: [(Array someInstance allSatisfy: [:e| e == nil or: [e isBehavior]])
 and: [(firstClassTablePage first: 17) asSet = {nil. SmallInteger.
Character. Array} asSet]]) ifTrue:
[| firstClassTablePage clone |
firstClassTablePage := Array someInstance.
self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e
isBehavior]]).
self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger.
Character. Array} asSet.
firstClassTablePage at: 4 put: SmallFloat64.
SmallFloat64 tryPrimitive: 161 withArgs: #(3).
clone := Array shallowCopy.
Array adoptInstance: clone.
clone tryPrimitive: 161 withArgs: #(16).
clone tryPrimitive: 160 withArgs: {Array someInstance}
self assert: SmallFloat64 identityHash = 3.
self assert: (Array someInstance first: 4) = {nil. SmallInteger. Character.
SmallFloat64}]

Then create a second update map to ensure that that version of Kernel is
loaded.

I will do this for Squeak immediately.


Apologies.

P.S. Interestingly enough, it shows what a horrible security hole the
debugger primitives tryPrimitive:withArgs: and tryNamedPrimitive:withArgs:
are.

Cheers,
> -- Pavel
>

_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20160308/cdb0d036/attachment.htm