Ok, I'm confused about the comments in initialCleanup, since in testing with a Pharo image from feb it DOES call the flushExternalPrimitives, but the comments imply a 3.8 or later image shouldn't have the root bit set.
Also then the flushExternalPrimitiveOf: seems to be invoked by the snapshotCleanUp.
So is the comment in initialCleanup wrong?
Could in the Closure VM could we skip flushExternalPrimitives for the new image type, since we *know* any closure image would have been processed by snapshotCleanUp, versus being a 3.2 image?
initialCleanup "Images written by VMs earlier than 3.6/3.7 will wrongly have the root bit set on the active context. Besides clearing the root bit, we treat this as a marker that these images also lack a cleanup of external primitives (which has been introduced at the same time when the root bit problem was fixed). In this case, we merely flush them from here."
((self longAt: activeContext) bitAnd: RootBit) = 0 ifTrue:[^nil]. "root bit is clean" "Clean root bit of activeContext" self longAt: activeContext put: ((self longAt: activeContext) bitAnd: AllButRootBit). "Clean external primitives" self flushExternalPrimitives.
then
flushExternalPrimitives "Flush the references to external functions from plugin primitives. This will force a reload of those primitives when accessed next. Note: We must flush the method cache here so that any failed primitives are looked up again." | oop primIdx | oop := self firstObject. [self oop: oop isLessThan: endOfMemory] whileTrue: [(self isFreeObject: oop) ifFalse: [(self isCompiledMethod: oop) ifTrue: ["This is a compiled method" primIdx := self primitiveIndexOf: oop. primIdx = PrimitiveExternalCallIndex ifTrue: ["It's primitiveExternalCall" self flushExternalPrimitiveOf: oop]]]. oop := self objectAfter: oop]. self flushMethodCache. self flushObsoleteIndexedPrimitives. self flushExternalPrimitiveTable
snapshotCleanUp "Clean up right before saving an image, sweeping memory and: * nilling out all fields of contexts above the stack pointer. * flushing external primitives * clearing the root bit of any object in the root table " | oop header fmt sz | oop := self firstObject. [self oop: oop isLessThan: endOfMemory] whileTrue: [(self isFreeObject: oop) ifFalse: [header := self longAt: oop. fmt := header >> 8 bitAnd: 15. "Clean out context" (fmt = 3 and: [self isContextHeader: header]) ifTrue: [sz := self sizeBitsOf: oop. (self lastPointerOf: oop) + BytesPerWord to: sz - BaseHeaderSize by: BytesPerWord do: [:i | self longAt: oop + i put: nilObj]]. "Clean out external functions" fmt >= 12 ifTrue: ["This is a compiled method" (self primitiveIndexOf: oop) = PrimitiveExternalCallIndex ifTrue: ["It's primitiveExternalCall" self flushExternalPrimitiveOf: oop]]]. oop := self objectAfter: oop]. self clearRootsTable
-- = = = ======================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ========================================================================
On Sat, Apr 11, 2009 at 11:55 PM, John M McIntosh < johnmci@smalltalkconsulting.com> wrote:
Ok, I'm confused about the comments in initialCleanup, since in testing with a Pharo image from feb it DOES call the flushExternalPrimitives, but the comments imply a 3.8 or later image shouldn't have the root bit set.
Also then the flushExternalPrimitiveOf: seems to be invoked by the snapshotCleanUp.
So is the comment in initialCleanup wrong?
Could in the Closure VM could we skip flushExternalPrimitives for the new image type, since we *know* any closure image would have been processed by snapshotCleanUp, versus being a 3.2 image?
Good point. So initialCleanup would then become
flushExternalPrimitives self flushMethodCache. self flushAtCache. self flushExternalPrimitiveTable
initialCleanup "Images written by VMs earlier than 3.6/3.7 will wrongly have the root bit set on the active context. Besides clearing the root bit, we treat this as a marker that these images also lack a cleanup of external primitives (which has been introduced at the same time when the root bit problem was fixed). In this case, we merely flush them from here."
((self longAt: activeContext) bitAnd: RootBit) = 0 ifTrue:[^nil].
"root bit is clean" "Clean root bit of activeContext" self longAt: activeContext put: ((self longAt: activeContext) bitAnd: AllButRootBit). "Clean external primitives" self flushExternalPrimitives.
then
flushExternalPrimitives "Flush the references to external functions from plugin primitives. This will force a reload of those primitives when accessed next. Note: We must flush the method cache here so that any failed primitives are looked up again." | oop primIdx | oop := self firstObject. [self oop: oop isLessThan: endOfMemory] whileTrue: [(self isFreeObject: oop) ifFalse: [(self isCompiledMethod: oop) ifTrue: ["This is a compiled method" primIdx := self primitiveIndexOf: oop. primIdx = PrimitiveExternalCallIndex ifTrue: ["It's primitiveExternalCall" self flushExternalPrimitiveOf: oop]]]. oop := self objectAfter: oop]. self flushMethodCache. self flushObsoleteIndexedPrimitives. self flushExternalPrimitiveTable
snapshotCleanUp "Clean up right before saving an image, sweeping memory and: * nilling out all fields of contexts above the stack pointer. * flushing external primitives * clearing the root bit of any object in the root table " | oop header fmt sz | oop := self firstObject. [self oop: oop isLessThan: endOfMemory] whileTrue: [(self isFreeObject: oop) ifFalse: [header := self longAt: oop. fmt := header >> 8 bitAnd: 15. "Clean out context" (fmt = 3 and: [self isContextHeader: header]) ifTrue: [sz := self sizeBitsOf: oop. (self lastPointerOf: oop) + BytesPerWord to: sz - BaseHeaderSize by: BytesPerWord do: [:i | self longAt: oop + i put: nilObj]]. "Clean out external functions" fmt >= 12 ifTrue: ["This is a compiled method" (self primitiveIndexOf: oop) = PrimitiveExternalCallIndex ifTrue: ["It's primitiveExternalCall" self flushExternalPrimitiveOf: oop]]]. oop := self objectAfter: oop]. self clearRootsTable
--
John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
On Mon, Apr 13, 2009 at 2:08 PM, Eliot Miranda eliot.miranda@gmail.comwrote:
On Sat, Apr 11, 2009 at 11:55 PM, John M McIntosh < johnmci@smalltalkconsulting.com> wrote:
Ok, I'm confused about the comments in initialCleanup, since in testing with a Pharo image from feb it DOES call the flushExternalPrimitives, but the comments imply a 3.8 or later image shouldn't have the root bit set.
Also then the flushExternalPrimitiveOf: seems to be invoked by the snapshotCleanUp.
So is the comment in initialCleanup wrong?
Could in the Closure VM could we skip flushExternalPrimitives for the new image type, since we *know* any closure image would have been processed by snapshotCleanUp, versus being a 3.2 image?
Good point. So initialCleanup would then become
flushExternalPrimitives self flushMethodCache. self flushAtCache. self flushExternalPrimitiveTable
Oops; copy paste error. I meant of course
initialCleanup self flushMethodCache. self flushAtCache. self flushExternalPrimitiveTable
initialCleanup "Images written by VMs earlier than 3.6/3.7 will wrongly have the root bit set on the active context. Besides clearing the root bit, we treat this as a marker that these images also lack a cleanup of external primitives (which has been introduced at the same time when the root bit problem was fixed). In this case, we merely flush them from here."
((self longAt: activeContext) bitAnd: RootBit) = 0 ifTrue:[^nil].
"root bit is clean" "Clean root bit of activeContext" self longAt: activeContext put: ((self longAt: activeContext) bitAnd: AllButRootBit). "Clean external primitives" self flushExternalPrimitives.
then
flushExternalPrimitives "Flush the references to external functions from plugin primitives. This will force a reload of those primitives when accessed next. Note: We must flush the method cache here so that any failed primitives are looked up again." | oop primIdx | oop := self firstObject. [self oop: oop isLessThan: endOfMemory] whileTrue: [(self isFreeObject: oop) ifFalse: [(self isCompiledMethod: oop) ifTrue: ["This is a compiled method" primIdx := self primitiveIndexOf: oop. primIdx = PrimitiveExternalCallIndex ifTrue: ["It's primitiveExternalCall"
self flushExternalPrimitiveOf: oop]]]. oop := self objectAfter: oop]. self flushMethodCache. self flushObsoleteIndexedPrimitives. self flushExternalPrimitiveTable
snapshotCleanUp "Clean up right before saving an image, sweeping memory and: * nilling out all fields of contexts above the stack pointer. * flushing external primitives * clearing the root bit of any object in the root table " | oop header fmt sz | oop := self firstObject. [self oop: oop isLessThan: endOfMemory] whileTrue: [(self isFreeObject: oop) ifFalse: [header := self longAt: oop. fmt := header >> 8 bitAnd: 15. "Clean out context" (fmt = 3 and: [self isContextHeader: header]) ifTrue: [sz := self sizeBitsOf: oop. (self lastPointerOf: oop) + BytesPerWord to: sz - BaseHeaderSize by: BytesPerWord do: [:i | self longAt: oop + i put: nilObj]]. "Clean out external functions" fmt >= 12 ifTrue: ["This is a compiled method" (self primitiveIndexOf: oop) = PrimitiveExternalCallIndex ifTrue: ["It's primitiveExternalCall"
self flushExternalPrimitiveOf: oop]]]. oop := self objectAfter: oop]. self clearRootsTable
--
=========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
===========================================================================
Er I think my question was why is
((self longAt: activeContext) bitAnd: RootBit) = 0 ifTrue:[
always false if the comment implies 1 3.8 shouldn't have the RootBit set?
and
flushExternalPrimitives then grinds thru all the objects to do self flushExternalPrimitiveOf: oop but snapshotCleanUp does the same work (or so it appears), which would mean the flushExternalPrimitiveOf: would be pointless at startup time.
On 13-Apr-09, at 2:09 PM, Eliot Miranda wrote:
--
= = = ======================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ========================================================================
On Mon, Apr 13, 2009 at 2:22 PM, John M McIntosh < johnmci@smalltalkconsulting.com> wrote:
Er I think my question was why is
((self longAt: activeContext) bitAnd: RootBit) = 0 ifTrue:[
always false if the comment implies 1 3.8 shouldn't have the RootBit set?
Since the initial context shouldn't have the root bit set all the context manipulation in initialClanup is redundant if this is a new closure image. If it isn't you need to keep it for loading older images. So in our sources the root bit code is still there in the ordinary closure interpreter, but gone in the Stack VM (which only runs closure images).
and
flushExternalPrimitives then grinds thru all the objects to do self flushExternalPrimitiveOf: oop but snapshotCleanUp does the same work (or so it appears), which would mean the flushExternalPrimitiveOf: would be pointless at startup time.
Right, which was why I said "Good point". In the Stack VM therefore initialCleanup is just
initialCleanup "This used to cope with issues with images written by VMs earlier than 3.6/3.7. Since we won't be loading such images (being a closure only VM) we only have to deal with external primitives. Since references to external plugins in methods are cleaned up in snapshotCleanUp only initialize the tables, not visit each method." self flushMethodCache. self flushAtCache. self flushExternalPrimitiveTable
On 13-Apr-09, at 2:09 PM, Eliot Miranda wrote:
--
=========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
vm-dev@lists.squeakfoundation.org