<div dir="ltr">Hi Pavel,<div><br></div><div> here's the debugged script that works in Squeak. It needs to be the *postload* script. I didn't realise but the preamble script is only run before a package is loaded, and is /not/ run before a package is updated.</div><div><br></div><div><div>"below, add code to be run after the loading of this package"</div><div><br></div><div>"Make sure that</div><div> a) SmallFloat64 has identityHash 3 and occupies the 4th (3rd, zero-relative) slot of the first class table page, and</div><div> b) that the first class table page is hidden by ensuring its class index is 16, the Array class index pun."</div><div>(Array someInstance size = 1024</div><div> and: [(Array someInstance allSatisfy: [:e| e == nil or: [e isBehavior]])</div><div> and: [(Array someInstance first: 17) asSet = {nil. SmallInteger. Character. Array} asSet]]) ifTrue:</div><div><span class="" style="white-space:pre"> </span>[| firstClassTablePage clone |</div><div><span class="" style="white-space:pre"> </span>firstClassTablePage := Array someInstance.</div><div><span class="" style="white-space:pre"> </span>self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e isBehavior]]).</div><div><span class="" style="white-space:pre"> </span>self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet.</div><div><span class="" style="white-space:pre"> </span>firstClassTablePage at: 4 put: SmallFloat64.</div><div><span class="" style="white-space:pre"> </span>SmallFloat64 tryPrimitive: 161 withArgs: #(3).</div><div><span class="" style="white-space:pre"> </span>clone := Array shallowCopy.</div><div><span class="" style="white-space:pre"> </span>Array adoptInstance: clone.</div><div><span class="" style="white-space:pre"> </span>clone tryPrimitive: 161 withArgs: #(16).</div><div><span class="" style="white-space:pre"> </span>clone tryPrimitive: 160 withArgs: {firstClassTablePage}.</div><div><span class="" style="white-space:pre"> </span>self assert: SmallFloat64 identityHash = 3.</div><div><span class="" style="white-space:pre"> </span>self assert: (firstClassTablePage first: 4) = {nil. SmallInteger. Character. SmallFloat64}]</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 8, 2016 at 10:38 AM, Eliot Miranda <span dir="ltr"><<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Pavel,<div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Tue, Mar 8, 2016 at 1:40 AM, Pavel Krivanek <span dir="ltr"><<a href="mailto:pavel.krivanek@gmail.com" target="_blank">pavel.krivanek@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Hi, <div><br></div><div>there is a strange array of size 1024 in the Pharo image that is not eaten by garbage collector even if no object points to it. I guess it is related to Spur.</div><div><br></div><div><div>((Array allInstances select: [ :i | i size = 1024 ])</div><div> <span style="white-space:pre-wrap"> </span>select: [ :a | a second = SmallInteger ]) pointersTo</div></div><div><br></div><div>What is it and is it accessible somehow?</div></div></blockquote><div><br></div></span><div>It is indeed the first class table page. It looks like an old bug. The oldest Spur image I can start up has the problem. The way that class table pages are hidden is that they have a class index which is not that of Array. Array is at index 52 in the class table (51 zero-relative, Array identityHash = 51), but it is also at index 17 (16 zero-relative). Array allInstances looks for objects whose class index is 51, hence not seeing the class table pages, which all (but the first) have class index 16.</div><div><br></div><div>Luckily we can fix the problem easily, and we can fix another problem at the same time, which is that SmallFloat64 has the wrong hash.</div><div><br></div><div>The class table is two-level; a (hidden) array of Arrays. Classes are in the class table at their identityHashes (so that to instantiate a class one copies the class's identity hash into the classIndex of the new instance instead of having to search the class table). But when SmallFloat64 was created in the 32-bit image I forgot to give it the right hash and store it in the class table. (there are no SmallFloat64 instances in a 32-bit image, only in 64-bits).</div><div><br></div><div>So we can first change SmallFloat64's hash to 3, which is what the Spur 64-bit VM and image require, and enter it into the first class table page, and then we can make the first class table page disappear.</div><div><br></div><div><div><br></div><div>| firstClassTablePage clone |</div><div>"The first class table page is the first Array instance. Of course this is a bug."</div><div>firstClassTablePage := Array someInstance.</div><div><br></div><div>"It should contain only nil and classes."</div><div>self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e isBehavior]]).</div><div><br></div><div>"And its first 17 elements should only be the immediate classes and Array, since Array is used as the class pun for class table pages, with index 17 (16 zero-relative)"</div><div>self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet.</div><div><br></div><div><div>"It just so happens that the second Array is the specialSelectors"</div><div>self assert: Array someInstance nextInstance == Smalltalk specialSelectors.</div></div><div><br></div><div>"Store SmallFloat64 in the first class table page at index 4 (3 zero-relative)."</div><div>firstClassTablePage at: 4 put: SmallFloat64.</div><div><br></div><div>"Use the secret set identity hash primitive to set SmallFloat64's identityHash to 3."</div><div>SmallFloat64 tryPrimitive: 161 withArgs: #(3).</div><div><br></div><div>"Now create an object that looks like Array class, but has identityHash 16."</div><div>"Take a shallow copy of Array class."</div><div>clone := Array shallowCopy.</div><div>"Change it into an Array."</div><div>Array adoptInstance: clone.</div><div>"Set its identityHash to 16."</div><div>clone tryPrimitive: 161 withArgs: #(16).</div><div>"Use the adoptInstance: primitive to ``set the class of the firstClassTablePage to the cone''.</div><div> or, more accurately, to set the firstClassTablePage's class index to 16."</div><div>clone tryPrimitive: 160 withArgs: {firstClassTablePage}</div></div><div><br></div><div><br></div><div>With the above done, we can check that everything is ok."</div>self assert: SmallFloat64 identityHash = 3.<div>self assert: Array someInstance == Smalltalk specialSelectors.</div><div>"A class table page is 1024 slots, contains only nil or behaviours, and contains at least one behaviour (is not all nils). There shouldn't be any that we can find."</div><div>self assert: (self systemNavigation allObjects select: [:o| o isArray and: [o size = 1024 and: [(o allSatisfy: [:e| e == nil or: [e isBehavior]]) and: [o anySatisfy: [:e| e isBehavior]]]]]) size = 0</div><div><br></div><div><br></div><div>I recommend you create an update map. Then create a version of kernel with a post-load action, written something like this:</div><div><br></div><div><div><br></div><div>(Array someInstance size = 1014</div><div> and: [(Array someInstance allSatisfy: [:e| e == nil or: [e isBehavior]])</div><div> and: [(firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet]]) ifTrue:</div><div><span style="white-space:pre-wrap"> </span>[| firstClassTablePage clone |</div><div><span style="white-space:pre-wrap"> </span>firstClassTablePage := Array someInstance.</div><div><span style="white-space:pre-wrap"> </span>self assert: (firstClassTablePage allSatisfy: [:e| e == nil or: [e isBehavior]]).</div><div><span style="white-space:pre-wrap"> </span>self assert: (firstClassTablePage first: 17) asSet = {nil. SmallInteger. Character. Array} asSet.</div><div><span style="white-space:pre-wrap"> </span>firstClassTablePage at: 4 put: SmallFloat64.</div><div><span style="white-space:pre-wrap"> </span>SmallFloat64 tryPrimitive: 161 withArgs: #(3).</div><div><span style="white-space:pre-wrap"> </span>clone := Array shallowCopy.</div><div><span style="white-space:pre-wrap"> </span>Array adoptInstance: clone.</div><div><span style="white-space:pre-wrap"> </span>clone tryPrimitive: 161 withArgs: #(16).</div><div><span style="white-space:pre-wrap"> </span>clone tryPrimitive: 160 withArgs: {Array someInstance}</div><div><span style="white-space:pre-wrap"> </span>self assert: SmallFloat64 identityHash = 3.</div><div><span style="white-space:pre-wrap"> </span>self assert: (Array someInstance first: 4) = {nil. SmallInteger. Character. SmallFloat64}]</div></div><div><br></div><div>Then create a second update map to ensure that that version of Kernel is loaded.</div><div><br></div><div>I will do this for Squeak immediately.</div><div><br></div><div><br></div><div>Apologies.</div><div><br></div><div>P.S. Interestingly enough, it shows what a horrible security hole the debugger primitives tryPrimitive:withArgs: and tryNamedPrimitive:withArgs: are.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Cheers,<br></div><div>-- Pavel</div></div>
</blockquote></div><div><br></div><div><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div>
</div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div>
</div>