Vm-dev January 2015

vm-dev@lists.squeakfoundation.org

22 participants
111 discussions

VM Maker: VMMaker.oscog-eem.1013.mcz
by commits＠source.squeak.org 09 Jan '15

09 Jan '15

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.1013.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.1013 Author: eem Time: 9 January 2015, 11:58:14.867 am UUID: 04aee5a6-0821-468f-be83-9f4a10199c94 Ancestors: VMMaker.oscog-eem.1012 Spur: Get following of the saved and caller contexts in base frames correct in followForwardingPointersInStackZone: General: Bow to the inevitable and add eassert: for those expensive asserts you just can't let go of. =============== Diff against VMMaker.oscog-eem.1012 =============== Item was changed: ----- Method: CCodeGenerator>>isAssertSelector: (in category 'inlining') ----- isAssertSelector: selector + ^#(assert: asserta: assert:l: asserta:l: deny: eassert:) includes: selector! - ^#(assert: asserta: assert:l: asserta:l: deny:) includes: selector! Item was changed: ----- Method: CoInterpreter>>followForwardingPointersInStackZone: (in category 'object memory support') ----- followForwardingPointersInStackZone: theBecomeEffectsFlags "Spur's become: is lazy, turning the becommed object into a forwarding object to the other. The read-barrier is minimised by arranging that forwarding pointers will fail a method cache probe, since notionally objects' internals are accessed only via sending messages to them, the exception is primitives that access the internals of the non-receiver argument(s). To avoid a read barrier on inst var fetch we scan the receivers in the stack zone and follow any forwarded ones. This is way cheaper than scanning all of memory as in the old become." | theIPPtr | <inline: false> <var: #thePage type: #'StackPage *'> <var: #theSP type: #'char *'> <var: #theFP type: #'char *'> <var: #callerFP type: #'char *'> <var: #theIPPtr type: #usqInt> (theBecomeEffectsFlags anyMask: BecameCompiledMethodFlag) ifTrue: [(objectMemory isForwarded: method) ifTrue: [theIPPtr := instructionPointer - method. method := objectMemory followForwarded: method. instructionPointer := method + theIPPtr]. (objectMemory isForwarded: newMethod) ifTrue: [newMethod := objectMemory followForwarded: newMethod]]. self assert: stackPage ~= 0. 0 to: numStackPages - 1 do: [:i| | thePage theSP theFP callerFP oop offset | thePage := stackPages stackPageAt: i. thePage isFree ifFalse: [theSP := thePage headSP. theFP := thePage headFP. "Skip the instruction pointer on top of stack of inactive pages." thePage = stackPage ifTrue: [theIPPtr := 0] ifFalse: [theIPPtr := theSP asUnsignedInteger. theSP := theSP + objectMemory wordSize]. [self assert: (thePage addressIsInPage: theFP). self assert: (theIPPtr = 0 or: [thePage addressIsInPage: theIPPtr asVoidPointer]). offset := self frameStackedReceiverOffset: theFP. oop := stackPages longAt: theFP + offset. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theFP + offset put: (objectMemory followForwarded: oop)]. ((self frameHasContext: theFP) and: [(objectMemory isForwarded: (self frameContext: theFP))]) ifTrue: [stackPages longAt: theFP + FoxThisContext put: (objectMemory followForwarded: (self frameContext: theFP))]. (self isMachineCodeFrame: theFP) ifTrue: [oop := stackPages longAt: theFP + FoxMFReceiver. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theFP + FoxMFReceiver put: (objectMemory followForwarded: oop)]. oop := (self mframeHomeMethod: theFP) methodObject. self assert: (objectMemory isForwarded: oop) not] ifFalse: [oop := stackPages longAt: theFP + FoxIFReceiver. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theFP + FoxIFReceiver put: (objectMemory followForwarded: oop)]. oop := self iframeMethod: theFP. (objectMemory isForwarded: oop) ifTrue: [| newOop delta | newOop := objectMemory followForwarded: oop. delta := newOop - oop. (theIPPtr ~= 0 and: [(stackPages longAt: theIPPtr) > oop]) ifTrue: [stackPages longAt: theIPPtr put: (stackPages longAt: theIPPtr) + delta]. stackPages longAt: theFP + FoxIFSavedIP put: (stackPages longAt: theFP + FoxIFSavedIP) + delta. stackPages longAt: theFP + FoxMethod put: (oop := newOop)]]. (callerFP := self frameCallerFP: theFP) ~= 0] whileTrue: [theIPPtr := (theFP + FoxCallerSavedIP) asUnsignedInteger. theFP := callerFP]. + "And finally follow the saved context and the caller context." + theSP := thePage baseAddress - objectMemory wordSize. - theSP := theFP + FoxCallerSavedIP + objectMemory wordSize. [theSP <= thePage baseAddress] whileTrue: [oop := stackPages longAt: theSP. (objectMemory isForwarded: oop) ifTrue: [stackPages longAt: theSP put: (objectMemory followForwarded: oop)]. theSP := theSP + objectMemory wordSize]]]! Item was changed: ----- Method: Spur32BitMMLECoSimulator>>testObjStackDo (in category 'ad-hoc tests') ----- testObjStackDo | size them seqA seqB seqC rs | + ExpensiveAsserts := true. self initializeWeaklingStack; emptyObjStack: weaklingStack. self assert: (self topOfObjStack: weaklingStack) isNil. self assert: (self capacityOfObjStack: weaklingStack) >= ObjStackLimit. seqA := (1 to: ObjStackLimit * 5 // 2) collect: [:i| self integerObjectOf: i]. seqA do: [:it| self noCheckPush: it onObjStack: weaklingStack]. them := Set new. size := self objStack: weaklingStack from: 0 do: [:it| them add: it]. self assert: size = seqA size. self assert: (them asSortedCollection asArray = seqA). self assert: (self isValidObjStack: weaklingStack). seqB := (ObjStackLimit * 5 // 2 + 1 to: ObjStackLimit * 10 // 2) collect: [:i| self integerObjectOf: i]. self assert: seqA size = seqB size. rs := seqB readStream. them := Set new. size := self objStack: weaklingStack from: 0 do: [:it| them add: it. self noCheckPush: rs next onObjStack: weaklingStack]. self assert: size = seqA size. self assert: rs atEnd. self objStack: weaklingStack from: size do: [:it| them add: it]. seqC := (seqA, seqB) sort. self assert: them asSortedCollection asArray = seqC! Item was changed: ----- Method: Spur32BitMMLESimulator>>testObjStackDo (in category 'ad-hoc tests') ----- testObjStackDo | size them seqA seqB seqC rs | + ExpensiveAsserts := true. self initializeWeaklingStack; emptyObjStack: weaklingStack. self assert: (self topOfObjStack: weaklingStack) isNil. self assert: (self capacityOfObjStack: weaklingStack) >= ObjStackLimit. seqA := (1 to: ObjStackLimit * 5 // 2) collect: [:i| self integerObjectOf: i]. seqA do: [:it| self noCheckPush: it onObjStack: weaklingStack]. them := Set new. size := self objStack: weaklingStack from: 0 do: [:it| them add: it]. self assert: size = seqA size. self assert: (them asSortedCollection asArray = seqA). self assert: (self isValidObjStack: weaklingStack). seqB := (ObjStackLimit * 5 // 2 + 1 to: ObjStackLimit * 10 // 2) collect: [:i| self integerObjectOf: i]. self assert: seqA size = seqB size. rs := seqB readStream. them := Set new. size := self objStack: weaklingStack from: 0 do: [:it| them add: it. self noCheckPush: rs next onObjStack: weaklingStack]. self assert: size = seqA size. self assert: rs atEnd. self objStack: weaklingStack from: size do: [:it| them add: it]. seqC := (seqA, seqB) sort. self assert: them asSortedCollection asArray = seqC! Item was changed: ----- Method: SpurMemoryManager>>emptyObjStack: (in category 'obj stacks') ----- emptyObjStack: objStack "Remove all the entries on the stack. Do so by setting Topx to 0 on the first page, and adding all subsequent pages to the free list." | nextPage nextNextPage | objStack = nilObj ifTrue: [^self]. + self eassert: (self isValidObjStack: objStack). - self assert: (self isValidObjStack: objStack). self storePointer: ObjStackTopx ofObjStack: objStack withValue: 0. nextPage := self fetchPointer: ObjStackNextx ofObject: objStack. [nextPage ~= 0] whileTrue: [nextNextPage := self fetchPointer: ObjStackNextx ofObject: nextPage. self storePointer: ObjStackFreex ofObjStack: nextPage withValue: (self fetchPointer: ObjStackFreex ofObject: objStack). self storePointer: ObjStackNextx ofObjStack: nextPage withValue: 0. self storePointer: ObjStackFreex ofObjStack: objStack withValue: nextPage. nextPage := nextNextPage]. self storePointer: ObjStackNextx ofObjStack: objStack withValue: 0. + self eassert: (self isValidObjStack: objStack)! - self assert: (self isValidObjStack: objStack)! Item was changed: ----- Method: SpurMemoryManager>>isEmptyObjStack: (in category 'obj stacks') ----- isEmptyObjStack: objStack objStack = nilObj ifTrue: [^true]. + self eassert: (self isValidObjStack: objStack). - self assert: (self isValidObjStack: objStack). ^0 = (self fetchPointer: ObjStackTopx ofObject: objStack) and: [0 = (self fetchPointer: ObjStackNextx ofObject: objStack)]! Item was changed: ----- Method: SpurMemoryManager>>noCheckPush:onObjStack: (in category 'obj stacks') ----- noCheckPush: objOop onObjStack: objStack "Push an element on an objStack. Split from push:onObjStack: for testing." | topx | + self eassert: (self isValidObjStack: objStack). - self assert: (self isValidObjStack: objStack). self cCode: [] "for debugging markAndTrace: set (MarkStackRecord := OrderedCollection new)" inSmalltalk: [(self fetchPointer: ObjStackMyx ofObject: objStack) = MarkStackRootIndex ifTrue: [MarkStackRecord ifNotNil: [MarkStackRecord addLast: {#push. objOop}]]]. topx := self fetchPointer: ObjStackTopx ofObject: objStack. topx >= ObjStackLimit ifTrue: [self noCheckPush: objOop onObjStack: (self ensureRoomOnObjStackAt: (self fetchPointer: ObjStackMyx ofObject: objStack))] ifFalse: [self storePointer: ObjStackFixedSlots + topx ofObjStack: objStack withValue: objOop. self storePointer: ObjStackTopx ofObjStack: objStack withValue: topx + 1]. ^objOop! Item was changed: ----- Method: SpurMemoryManager>>objStack:from:do: (in category 'obj stacks') ----- objStack: objStack from: start do: aBlock "Evaluate aBlock with all elements from start (0-relative) in objStack. Answer the size of the stack *before* the enumeration commences. This evaluates in top-of-stack-to-bottom order. N.B. this is also stable if aBlock causes new elements to be added to the objStack, but unstable if aBlock causes elements to be removed." <inline: true> | size objStackPage numToEnumerate | + self eassert: (self isValidObjStack: weaklingStack). - self assert: (self isValidObjStack: weaklingStack). size := self fetchPointer: ObjStackTopx ofObject: objStack. objStackPage := self fetchPointer: ObjStackNextx ofObject: objStack. [objStackPage ~= 0] whileTrue: [size := size + ObjStackLimit. self assert: (self fetchPointer: ObjStackTopx ofObject: objStackPage) = ObjStackLimit. objStackPage := self fetchPointer: ObjStackNextx ofObject: objStackPage]. numToEnumerate := size - start. objStackPage := objStack. [numToEnumerate > 0] whileTrue: [| numOnThisPage numToEnumerateOnThisPage topIndex | numOnThisPage := self fetchPointer: ObjStackTopx ofObject: objStackPage. numToEnumerateOnThisPage := numToEnumerate min: numOnThisPage. topIndex := numOnThisPage + ObjStackFixedSlots - 1. topIndex to: topIndex - numToEnumerateOnThisPage + 1 by: -1 do: [:i| self assert: (self isWeak: (self fetchPointer: i ofObject: objStackPage)). aBlock value: (self fetchPointer: i ofObject: objStackPage)]. numToEnumerate := numToEnumerate - numToEnumerateOnThisPage. objStackPage := self fetchPointer: ObjStackNextx ofObject: objStackPage]. ^size! Item was changed: ----- Method: SpurMemoryManager>>popObjStack: (in category 'obj stacks') ----- popObjStack: objStack | topx top nextPage myx | + self eassert: (self isValidObjStack: objStack). - self assert: (self isValidObjStack: objStack). topx := self fetchPointer: ObjStackTopx ofObject: objStack. topx = 0 ifTrue: [self assert: (self fetchPointer: ObjStackNextx ofObject: objStack) = 0. self cCode: [] "for debugging markAndTrace: set (MarkStackRecord := OrderedCollection new)" inSmalltalk: [(self fetchPointer: ObjStackMyx ofObject: objStack) = MarkStackRootIndex ifTrue: [MarkStackRecord ifNotNil: [MarkStackRecord addLast: {#EMPTY. nil}]]]. ^nil]. topx := topx - 1. top := self fetchPointer: topx + ObjStackFixedSlots ofObject: objStack. self cCode: [] "for debugging markAndTrace: set (MarkStackRecord := OrderedCollection new)" inSmalltalk: [(self fetchPointer: ObjStackMyx ofObject: objStack) = MarkStackRootIndex ifTrue: [MarkStackRecord ifNotNil: [(MarkStackRecord last first = #push and: [MarkStackRecord last last = top]) ifTrue: [MarkStackRecord removeLast] ifFalse: [MarkStackRecord addLast: {#pop. top}]]]]. self storePointer: ObjStackTopx ofObjStack: objStack withValue: topx. (topx = 0 and: [(nextPage := self fetchPointer: ObjStackNextx ofObject: objStack) ~= 0]) ifTrue: [self storePointer: ObjStackFreex ofObjStack: nextPage withValue: objStack. self storePointer: ObjStackNextx ofObjStack: objStack withValue: 0. myx := self fetchPointer: ObjStackMyx ofObject: objStack. self updateRootOfObjStackAt: myx with: nextPage. + self eassert: (self isValidObjStack: nextPage)] - self assert: (self isValidObjStack: nextPage)] ifFalse: + [self eassert: (self isValidObjStack: objStack)]. - [self assert: (self isValidObjStack: objStack)]. ^top! Item was changed: ----- Method: StackInterpreter class>>mustBeGlobal: (in category 'translation') ----- mustBeGlobal: var "Answer if a variable must be global and exported. Used for inst vars that are accessed from VM support code." + ^(super mustBeGlobal: var) + or: [(self objectMemoryClass mustBeGlobal: var) - ^(self objectMemoryClass mustBeGlobal: var) or: [(#('interpreterProxy' 'interpreterVersion' 'inIOProcessEvents' 'deferDisplayUpdates' 'extraVMMemory' 'showSurfaceFn' 'desiredNumStackPages' 'desiredEdenBytes' 'breakSelector' 'breakSelectorLength' 'sendTrace' 'checkAllocFiller' 'suppressHeartbeatFlag') includes: var) or: [ "This allows slow machines to define bytecodeSetSelector as 0 to avoid the interpretation overhead." + MULTIPLEBYTECODESETS not and: [var = 'bytecodeSetSelector']]]]! - MULTIPLEBYTECODESETS not and: [var = 'bytecodeSetSelector']]]! Item was changed: ----- Method: StackInterpreter>>followForwardingPointersInStackZone: (in category 'object memory support') ----- followForwardingPointersInStackZone: theBecomeEffectsFlags "Spur's become: is lazy, turning the becommed object into a forwarding object to the other. The read-barrier is minimised by arranging that forwarding pointers will fail a method cache probe, since notionally objects' internals are accessed only via sending messages to them, the exception is primitives that access the internals of the non-receiver argument(s). To avoid a read barrier on bytecode, literal and inst var fetch we scan the receivers and methods in the stack zone and follow any forwarded ones. This is of course way cheaper than scanning all of memory as in the old become." | theIPPtr | <inline: false> <var: #thePage type: #'StackPage *'> <var: #theSP type: #'char *'> <var: #theFP type: #'char *'> <var: #callerFP type: #'char *'> <var: #theIPPtr type: #usqInt> (theBecomeEffectsFlags anyMask: BecameCompiledMethodFlag) ifTrue: [(objectMemory isForwarded: method) ifTrue: [theIPPtr := instructionPointer - method. method := objectMemory followForwarded: method. instructionPointer := method + theIPPtr]. (objectMemory isForwarded: newMethod) ifTrue: [newMethod := objectMemory followForwarded: newMethod]]. self assert: stackPage ~= 0. 0 to: numStackPages - 1 do: [:i| | thePage theSP theFP callerFP theIP oop | thePage := stackPages stackPageAt: i. thePage isFree ifFalse: [theSP := thePage headSP. theFP := thePage headFP. "Skip the instruction pointer on top of stack of inactive pages." thePage = stackPage ifTrue: [theIPPtr := 0] ifFalse: [theIPPtr := theSP asInteger. theSP := theSP + objectMemory wordSize]. [self assert: (thePage addressIsInPage: theFP). self assert: (theIPPtr = 0 or: [thePage addressIsInPage: theIPPtr asVoidPointer]). oop := stackPages longAt: theFP + FoxReceiver. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theFP + FoxReceiver put: (objectMemory followForwarded: oop)]. theIP := (theFP + (self frameStackedReceiverOffset: theFP)) asInteger. "reuse theIP; its just an offset here" oop := stackPages longAt: theIP. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theIP put: (objectMemory followForwarded: oop)]. ((self frameHasContext: theFP) and: [(objectMemory isForwarded: (self frameContext: theFP))]) ifTrue: [stackPages longAt: theFP + FoxThisContext put: (objectMemory followForwarded: (self frameContext: theFP))]. oop := self frameMethod: theFP. (objectMemory isForwarded: oop) ifTrue: [| newOop delta | newOop := objectMemory followForwarded: oop. theIPPtr ~= 0 ifTrue: [self assert: (stackPages longAt: theIPPtr) > (self frameMethod: theFP). delta := newOop - oop. stackPages longAt: theIPPtr put: (stackPages longAt: theIPPtr) + delta]. stackPages longAt: theFP + FoxMethod put: (oop := newOop)]. (callerFP := self frameCallerFP: theFP) ~= 0] whileTrue: [theIPPtr := (theFP + FoxCallerSavedIP) asInteger. theFP := callerFP]. + "And finally follow the caller context." + self assert: theFP = thePage baseFP. + oop := self frameCallerContext: theFP. + (objectMemory isForwarded: oop) ifTrue: + [self frameCallerContext: theFP put: (objectMemory followForwarded: oop)]]]! - theSP := theFP + FoxCallerContext. "a.k.a. FoxCallerSavedIP" - [theSP <= thePage baseAddress] whileTrue: - [oop := stackPages longAt: theSP. - (objectMemory isForwarded: oop) ifTrue: - [stackPages longAt: theSP put: (objectMemory followForwarded: oop)]. - theSP := theSP + objectMemory wordSize]]]! Item was changed: Object subclass: #VMClass instanceVariableNames: '' + classVariableNames: 'DefaultBase ExpensiveAsserts' - classVariableNames: 'DefaultBase' poolDictionaries: 'VMBasicConstants VMObjectIndices' category: 'VMMaker-Support'! VMClass class instanceVariableNames: 'timeStamp initializationOptions'! !VMClass commentStamp: '<historical>' prior: 0! I am an abstract superclass for all classes in the VM that want to maintain a source timeStamp.! VMClass class instanceVariableNames: 'timeStamp initializationOptions'! Item was changed: ----- Method: VMClass class>>declareCVarsIn: (in category 'translation') ----- declareCVarsIn: aCCodeGenerator + "Declare any additional variables and/or add type declarations for existing variables." + aCCodeGenerator + var: #expensiveAsserts + declareC: 'char expensiveAsserts = 0'! - "Declare any additional variables and/or add type declarations for existing variables."! Item was changed: ----- Method: VMClass class>>initialize (in category 'initialization') ----- initialize (Utilities classPool at: #CommonRequestStrings ifAbsent: []) ifNotNil: [:commonRequestStringHolder| (commonRequestStringHolder contents asString includesSubString: 'VMClass open') ifFalse: + [Utilities appendToCommonRequests: '-\VMMaker generateConfiguration\VMMaker generateAllConfigurationsUnderVersionControl\VMMaker generateAllSpurConfigurations\VMClass openCogMultiWindowBrowser\VMClass openObjectMemoriesInterpretersBrowser\VMClass openSpurMultiWindowBrowser' withCRs]]. + ExpensiveAsserts := false! - [Utilities appendToCommonRequests: '-\VMMaker generateConfiguration\VMMaker generateAllConfigurationsUnderVersionControl\VMMaker generateAllSpurConfigurations\VMClass openCogMultiWindowBrowser\VMClass openObjectMemoriesInterpretersBrowser\VMClass openSpurMultiWindowBrowser' withCRs]]! Item was changed: ----- Method: VMClass class>>initializeWithOptions: (in category 'initialization') ----- initializeWithOptions: optionsDictionaryOrArray "Initialize the receiver, typically initializing class variables. Initialize any class variables whose names occur in optionsDictionary with the corresponding values there-in." | optionsDictionary | optionsDictionary := optionsDictionaryOrArray isArray ifTrue: [Dictionary newFromPairs: optionsDictionaryOrArray] ifFalse: [optionsDictionaryOrArray]. "This is necessary. e.g. if the receiver is CoInterpreterPrimitives, it is still necessary to set the options in CoInterpreter. Otherwise, some class in the chain may have stale options, and when building the code generator, a stale ancilliary class may be computed." (self withAllSuperclasses copyUpThrough: VMClass) do: [:class| + class initializationOptions: optionsDictionary]. + + ExpensiveAsserts := optionsDictionary at: #ExpensiveAsserts ifAbsent: [false]! - class initializationOptions: optionsDictionary]! Item was added: + ----- Method: VMClass class>>mustBeGlobal: (in category 'translation') ----- + mustBeGlobal: var + "Answer if a variable must be global and exported. Used for inst vars that are accessed from VM support code." + + ^var = #expensiveAsserts! Item was added: + ----- Method: VMClass>>eassert: (in category 'debug support') ----- + eassert: aBooleanExpressionOrBlock + "This is for expensive asserts that we're only interested in checking in extremis. + For exampl,e now that Spur objStacks are debugged there's no benefit to evaluating + isValidObjStack: throguhout the mark loop because its damn slow." + <doNotGenerate> + ExpensiveAsserts ifTrue: + [aBooleanExpressionOrBlock value ifFalse: + [AssertionFailure signal: 'Assertion failed']]!

1 0

VM Maker: VMMaker.oscog-eem.1012.mcz
by commits＠source.squeak.org 09 Jan '15

09 Jan '15

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.1012.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.1012 Author: eem Time: 8 January 2015, 5:00:01.721 pm UUID: 0c6e99ae-1afa-4f1e-a20c-3ded152f264d Ancestors: VMMaker.oscog-eem.1011 Spur: Make sure the saved contexts at the top of each stack page are followed post-become in followForwardingPointersInStackZone:. Add an assert to Spur's shouldRemapObj: to verify that new space references are only visited once in a scavenge. Add asserts to check that contexts are not forwarded when marrying stable contexts. General: Never inline mapStackPages, for debugging. Add an assert to returnToExecutive:postContextSwitch: to ensure the instructionPointer is valid when checking for being at a send return pc. Eliminate the simulator overrides for withSmallIntegerTags: and don't assume the SmallInteger tag pattern. =============== Diff against VMMaker.oscog-eem.1011 =============== Item was changed: ----- Method: CoInterpreter>>followForwardingPointersInStackZone: (in category 'object memory support') ----- followForwardingPointersInStackZone: theBecomeEffectsFlags "Spur's become: is lazy, turning the becommed object into a forwarding object to the other. The read-barrier is minimised by arranging that forwarding pointers will fail a method cache probe, since notionally objects' internals are accessed only via sending messages to them, the exception is primitives that access the internals of the non-receiver argument(s). To avoid a read barrier on inst var fetch we scan the receivers in the stack zone and follow any forwarded ones. This is way cheaper than scanning all of memory as in the old become." | theIPPtr | <inline: false> <var: #thePage type: #'StackPage *'> <var: #theSP type: #'char *'> <var: #theFP type: #'char *'> <var: #callerFP type: #'char *'> <var: #theIPPtr type: #usqInt> (theBecomeEffectsFlags anyMask: BecameCompiledMethodFlag) ifTrue: [(objectMemory isForwarded: method) ifTrue: [theIPPtr := instructionPointer - method. method := objectMemory followForwarded: method. instructionPointer := method + theIPPtr]. (objectMemory isForwarded: newMethod) ifTrue: [newMethod := objectMemory followForwarded: newMethod]]. self assert: stackPage ~= 0. 0 to: numStackPages - 1 do: [:i| | thePage theSP theFP callerFP oop offset | thePage := stackPages stackPageAt: i. thePage isFree ifFalse: [theSP := thePage headSP. theFP := thePage headFP. "Skip the instruction pointer on top of stack of inactive pages." thePage = stackPage ifTrue: [theIPPtr := 0] ifFalse: [theIPPtr := theSP asUnsignedInteger. theSP := theSP + objectMemory wordSize]. [self assert: (thePage addressIsInPage: theFP). self assert: (theIPPtr = 0 or: [thePage addressIsInPage: theIPPtr asVoidPointer]). offset := self frameStackedReceiverOffset: theFP. oop := stackPages longAt: theFP + offset. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theFP + offset put: (objectMemory followForwarded: oop)]. ((self frameHasContext: theFP) and: [(objectMemory isForwarded: (self frameContext: theFP))]) ifTrue: [stackPages longAt: theFP + FoxThisContext put: (objectMemory followForwarded: (self frameContext: theFP))]. (self isMachineCodeFrame: theFP) ifTrue: [oop := stackPages longAt: theFP + FoxMFReceiver. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theFP + FoxMFReceiver put: (objectMemory followForwarded: oop)]. oop := (self mframeHomeMethod: theFP) methodObject. self assert: (objectMemory isForwarded: oop) not] ifFalse: [oop := stackPages longAt: theFP + FoxIFReceiver. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theFP + FoxIFReceiver put: (objectMemory followForwarded: oop)]. oop := self iframeMethod: theFP. (objectMemory isForwarded: oop) ifTrue: [| newOop delta | newOop := objectMemory followForwarded: oop. delta := newOop - oop. (theIPPtr ~= 0 and: [(stackPages longAt: theIPPtr) > oop]) ifTrue: [stackPages longAt: theIPPtr put: (stackPages longAt: theIPPtr) + delta]. stackPages longAt: theFP + FoxIFSavedIP put: (stackPages longAt: theFP + FoxIFSavedIP) + delta. stackPages longAt: theFP + FoxMethod put: (oop := newOop)]]. (callerFP := self frameCallerFP: theFP) ~= 0] whileTrue: [theIPPtr := (theFP + FoxCallerSavedIP) asUnsignedInteger. + theFP := callerFP]. + theSP := theFP + FoxCallerSavedIP + objectMemory wordSize. + [theSP <= thePage baseAddress] whileTrue: + [oop := stackPages longAt: theSP. + (objectMemory isForwarded: oop) ifTrue: + [stackPages longAt: theSP put: (objectMemory followForwarded: oop)]. + theSP := theSP + objectMemory wordSize]]]! - theFP := callerFP]]]! Item was changed: ----- Method: CoInterpreter>>makeBaseFrameFor: (in category 'frame access') ----- makeBaseFrameFor: aContext "<Integer>" "Marry aContext with the base frame of a new stack page. Build the base frame to reflect the context's state. Answer the new page. Override to hold the caller context in a different place, In the StackInterpreter we use the caller saved ip, but in the Cog VM caller saved ip is the ceBaseReturn: trampoline. Simply hold the caller context in the first word of the stack." <returnTypeC: #'StackPage *'> | page pointer theMethod theIP numArgs stackPtrIndex maybeClosure rcvr | <inline: false> <var: #page type: #'StackPage *'> <var: #pointer type: #'char *'> <var: #cogMethod type: #'CogMethod *'> "theIP must be typed as signed because it is assigned ceCannotResumePC and so maybe implicitly typed as unsigned." <var: #theIP type: #sqInt> + self assert: (objectMemory isContext: aContext). self assert: (self isSingleContext: aContext). self assert: (objectMemory goodContextSize: aContext). theIP := objectMemory fetchPointer: InstructionPointerIndex ofObject: aContext. self assert: HasBeenReturnedFromMCPC signedIntFromLong < 0. theIP := (objectMemory isIntegerObject: theIP) ifTrue: [objectMemory integerValueOf: theIP] ifFalse: [HasBeenReturnedFromMCPC]. theMethod := objectMemory followObjField: MethodIndex ofObject: aContext. page := self newStackPage. "first word on stack is caller context of base frame" stackPages longAt: (pointer := page baseAddress) put: (objectMemory fetchPointer: SenderIndex ofObject: aContext). "second word is the context itself; needed for cannotReturn processing; see ceBaseReturn:." stackPages longAt: (pointer := pointer - objectMemory wordSize) put: aContext. rcvr := objectMemory followField: ReceiverIndex ofObject: aContext. "If the frame is a closure activation then the closure should be on the stack in the pushed receiver position (closures receiver the value[:value:] messages). Otherwise it should be the receiver proper." maybeClosure := objectMemory fetchPointer: ClosureIndex ofObject: aContext. maybeClosure ~= objectMemory nilObject ifTrue: [numArgs := self argumentCountOfClosure: maybeClosure. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: maybeClosure] ifFalse: [| header | header := objectMemory methodHeaderOf: theMethod. numArgs := self argumentCountOfMethodHeader: header. "If this is a synthetic context its IP could be pointing at the CallPrimitive opcode. If so, skip it." (theIP signedIntFromLong > 0 and: [(self methodHeaderHasPrimitive: header) and: [theIP = (1 + (objectMemory lastPointerOf: theMethod))]]) ifTrue: [theIP := theIP + (self sizeOfCallPrimitiveBytecode: header)]. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: rcvr]. "Put the arguments on the stack" 1 to: numArgs do: [:i| stackPages longAt: (pointer := pointer - objectMemory wordSize) put: (objectMemory fetchPointer: ReceiverIndex + i ofObject: aContext)]. "saved caller ip is base return trampoline" stackPages longAt: (pointer := pointer - objectMemory wordSize) put: cogit ceBaseFrameReturnPC. "base frame's saved fp is null" stackPages longAt: (pointer := pointer - objectMemory wordSize) put: 0. "N.B. Don't set the baseFP, which marks the page as in use, until after ensureMethodIsCogged: and/or instructionPointer:forContext:frame:. These can cause a compiled code compaction which, if marked as in use, will examine this partially initialized page and crash." page headFP: pointer. "Create either a machine code frame or an interpreter frame based on the pc. If the pc is -ve it is a machine code pc and so we produce a machine code frame. If +ve an interpreter frame. N.B. Do *not* change this to try and map from a bytecode pc to a machine code frame under any circumstances. See ensureContextIsExecutionSafeAfterAssignToStackPointer:" theIP signedIntFromLong < 0 ifTrue: [| cogMethod | "Since we would have to generate a machine-code method to be able to map the native pc anyway we should create a native method and native frame." cogMethod := self ensureMethodIsCogged: theMethod. theMethod := cogMethod asInteger. maybeClosure ~= objectMemory nilObject ifTrue: ["If the pc is the special HasBeenReturnedFromMCPC pc set the pc appropriately so that the frame stays in the cannotReturn: state." theIP = HasBeenReturnedFromMCPC signedIntFromLong ifTrue: [theMethod := (cogit findMethodForStartBcpc: (self startPCOfClosure: maybeClosure) inHomeMethod: (self cCoerceSimple: theMethod to: #'CogMethod *')) asInteger. theMethod = 0 ifTrue: [self error: 'cannot find machine code block matching closure''s startpc']. theIP := cogit ceCannotResumePC] ifFalse: [self assert: (theIP signedBitShift: -16) < -1. "See contextInstructionPointer:frame:" theMethod := theMethod - ((theIP signedBitShift: -16) * cogit blockAlignment). theIP := theMethod - theIP signedIntFromShort]. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: theMethod + MFMethodFlagHasContextFlag + MFMethodFlagIsBlockFlag] ifFalse: [self assert: (theIP signedBitShift: -16) >= -1. "If the pc is the special HasBeenReturnedFromMCPC pc set the pc appropriately so that the frame stays in the cannotReturn: state." theIP := theIP = HasBeenReturnedFromMCPC signedIntFromLong ifTrue: [cogit ceCannotResumePC] ifFalse: [theMethod asInteger - theIP]. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: theMethod + MFMethodFlagHasContextFlag]. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: aContext] ifFalse: [stackPages longAt: (pointer := pointer - objectMemory wordSize) put: theMethod. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: aContext. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: (self encodeFrameFieldHasContext: true isBlock: maybeClosure ~= objectMemory nilObject numArgs: numArgs). stackPages longAt: (pointer := pointer - objectMemory wordSize) put: 0. "FoxIFSavedIP" theIP := self iframeInstructionPointerForIndex: theIP method: theMethod]. page baseFP: page headFP. self assert: (self frameHasContext: page baseFP). self assert: (self frameNumArgs: page baseFP) == numArgs. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: rcvr. stackPtrIndex := self quickFetchInteger: StackPointerIndex ofObject: aContext. self assert: ReceiverIndex + stackPtrIndex < (objectMemory lengthOf: aContext). numArgs + 1 to: stackPtrIndex do: [:i| stackPages longAt: (pointer := pointer - objectMemory wordSize) put: (objectMemory fetchPointer: ReceiverIndex + i ofObject: aContext)]. "top of stack is the instruction pointer" stackPages longAt: (pointer := pointer - objectMemory wordSize) put: theIP. page headSP: pointer. self assert: (self context: aContext hasValidInversePCMappingOf: theIP in: page baseFP). "Mark context as married by setting its sender to the frame pointer plus SmallInteger tags and the InstructionPointer to the saved fp (which ensures correct alignment w.r.t. the frame when we check for validity) plus SmallInteger tags." objectMemory storePointerUnchecked: SenderIndex ofObject: aContext withValue: (self withSmallIntegerTags: page baseFP). objectMemory storePointerUnchecked: InstructionPointerIndex ofObject: aContext withValue: (self withSmallIntegerTags: 0). self assert: (objectMemory isIntegerObject: (objectMemory fetchPointer: SenderIndex ofObject: aContext)). self assert: (self frameOfMarriedContext: aContext) = page baseFP. self assert: (self validStackPageBaseFrame: page). ^page! Item was changed: ----- Method: CoInterpreter>>mapStackPages (in category 'object memory support') ----- mapStackPages + <inline: #never> - <inline: false> <var: #thePage type: #'StackPage *'> <var: #theSP type: #'char *'> <var: #theFP type: #'char *'> <var: #frameRcvrOffset type: #'char *'> <var: #callerFP type: #'char *'> <var: #theIPPtr type: #'char *'> 0 to: numStackPages - 1 do: [:i| | thePage theSP theFP frameRcvrOffset callerFP theIPPtr theIP oop | thePage := stackPages stackPageAt: i. thePage isFree ifFalse: [theSP := thePage headSP. theFP := thePage headFP. "Skip the instruction pointer on top of stack of inactive pages." thePage = stackPage ifTrue: [theIPPtr := ((self isMachineCodeFrame: theFP) or: [(self iframeSavedIP: theFP) = 0]) ifTrue: [0] ifFalse: [theFP + FoxIFSavedIP]] ifFalse: [theIPPtr := theSP. theSP := theSP + objectMemory wordSize]. [self assert: (thePage addressIsInPage: theFP). self assert: (thePage addressIsInPage: theSP). self assert: (theIPPtr = 0 or: [thePage addressIsInPage: theIPPtr]). frameRcvrOffset := self frameReceiverOffset: theFP. [theSP <= frameRcvrOffset] whileTrue: [oop := stackPages longAt: theSP. (objectMemory shouldRemapOop: oop) ifTrue: [stackPages longAt: theSP put: (objectMemory remapObj: oop)]. theSP := theSP + objectMemory wordSize]. (self frameHasContext: theFP) ifTrue: [(objectMemory shouldRemapObj: (self frameContext: theFP)) ifTrue: [stackPages longAt: theFP + FoxThisContext put: (objectMemory remapObj: (self frameContext: theFP))]. "forwarding scheme in SqueakV3 obj rep makes this hard to check." objectMemory hasSpurMemoryManagerAPI ifTrue: [self assert: ((self isMarriedOrWidowedContext: (self frameContext: theFP)) and: [(self frameOfMarriedContext: (self frameContext: theFP)) = theFP])]]. (self isMachineCodeFrame: theFP) ifFalse: [(objectMemory shouldRemapObj: (self iframeMethod: theFP)) ifTrue: [theIPPtr ~= 0 ifTrue: [theIP := stackPages longAt: theIPPtr. theIP = cogit ceReturnToInterpreterPC ifTrue: [self assert: (self iframeSavedIP: theFP) > (self iframeMethod: theFP). theIPPtr := theFP + FoxIFSavedIP. theIP := stackPages longAt: theIPPtr] ifFalse: [self assert: theIP > (self iframeMethod: theFP)]. theIP := theIP - (self iframeMethod: theFP)]. stackPages longAt: theFP + FoxMethod put: (objectMemory remapObj: (self iframeMethod: theFP)). theIPPtr ~= 0 ifTrue: [stackPages longAt: theIPPtr put: theIP + (self iframeMethod: theFP)]]]. (callerFP := self frameCallerFP: theFP) ~= 0] whileTrue: [theSP := (theIPPtr := theFP + FoxCallerSavedIP) + objectMemory wordSize. theFP := callerFP]. theSP := theFP + FoxCallerSavedIP + objectMemory wordSize. [theSP <= thePage baseAddress] whileTrue: [oop := stackPages longAt: theSP. (objectMemory shouldRemapOop: oop) ifTrue: [stackPages longAt: theSP put: (objectMemory remapObj: oop)]. theSP := theSP + objectMemory wordSize]]]! Item was changed: ----- Method: CoInterpreter>>returnToExecutive:postContextSwitch: (in category 'enilopmarts') ----- returnToExecutive: inInterpreter postContextSwitch: switchedContext "Return to the current frame, either by entering machine code, or longjmp-ing back to the interpreter or simply returning, depending on where we are. To know whether to return or enter machine code we have to know from whence we came. We could have come from the interpreter, either directly or via a machine code primitive. We could have come from machine code. The instructionPointer tells us where from. If it is above startOfMemory we're in the interpreter. If it is below, then we are in machine-code unless it is ceReturnToInterpreterPC, in which case we're in a machine-code primitive called from the interpreter." <inline: false> | cogMethod retValue fullyInInterpreter | <var: #cogMethod type: #'CogBlockMethod *'> cogit assertCStackWellAligned. (self isMachineCodeFrame: framePointer) ifTrue: [self assertValidExecutionPointe: instructionPointer r: framePointer s: stackPointer imbar: false line: #'__LINE__'. "If returning after a context switch then a result may have to be popped from the stack. If the process is suspended at a send then the result of the primitive in which the process was suspended is still on the stack and must be popped into ReceiverResultReg. If not, nothing should be popped and ReceiverResultReg gets the receiver." switchedContext ifTrue: [cogMethod := self mframeCogMethod: framePointer. + self assert: (instructionPointer > cogit minCogMethodAddress + and: [instructionPointer < cogit maxCogMethodAddress]). (instructionPointer ~= (cogMethod asInteger + cogMethod stackCheckOffset) and: [cogit isSendReturnPC: instructionPointer]) ifTrue: [self assert: (objectMemory addressCouldBeOop: self stackTop). retValue := self popStack] ifFalse: [retValue := self mframeReceiver: framePointer]] ifFalse: [retValue := self mframeReceiver: framePointer]. self push: instructionPointer. self push: retValue. cogit ceEnterCogCodePopReceiverReg "NOTREACHED"]. self setMethod: (self iframeMethod: framePointer). fullyInInterpreter := inInterpreter. instructionPointer = cogit ceReturnToInterpreterPC ifTrue: [instructionPointer := (self iframeSavedIP: framePointer) asUnsignedInteger. fullyInInterpreter := false]. self assertValidExecutionPointe: instructionPointer r: framePointer s: stackPointer imbar: true line: #'__LINE__'. fullyInInterpreter ifFalse: [self siglong: reenterInterpreter jmp: ReturnToInterpreter. "NOTREACHED"]. ^nil! Item was removed: - ----- Method: CogVMSimulator>>withSmallIntegerTags: (in category 'frame access') ----- - withSmallIntegerTags: aValueOrSurrogate - "The simulator works with strictly positive bit patterns" - | value | - value := aValueOrSurrogate asInteger. - ^value < 0 - ifTrue: [(value bitAnd: 16rFFFFFFFF) + 1] - ifFalse: [value + 1]! Item was changed: ----- Method: InterpreterPrimitives>>asUnsigned: (in category 'primitive support') ----- asUnsigned: anInteger <inline: true> + ^self cCode: [anInteger asUnsignedLong] inSmalltalk: [anInteger bitAnd: objectMemory maxCInteger]! - ^self cCode: [anInteger asUnsignedLong] inSmalltalk: [anInteger bitAnd: objectMemory maxSmallInteger]! Item was added: + ----- Method: NewObjectMemory>>smallIntegerTag (in category 'cog jit support') ----- + smallIntegerTag + ^1! Item was added: + ----- Method: ObjectMemory>>maxCInteger (in category 'interpreter access') ----- + maxCInteger + ^16rFFFFFFFF! Item was added: + ----- Method: Spur32BitMemoryManager>>maxCInteger (in category 'interpreter access') ----- + maxCInteger + ^16rFFFFFFFF! Item was added: + ----- Method: Spur64BitMemoryManager>>maxCInteger (in category 'interpreter access') ----- + maxCInteger + ^16rFFFFFFFFFFFFFFFF! Item was added: + ----- Method: SpurMemoryManager>>maxCInteger (in category 'interpreter access') ----- + maxCInteger + self subclassResponsibility! Item was changed: ----- Method: SpurMemoryManager>>shouldRemapObj: (in category 'gc - scavenging') ----- shouldRemapObj: objOop <api> "Answer if the obj should be scavenged (or simply followed). The method is called shouldRemapObj: for compatibility with ObjectMemory." ^(self isForwarded: objOop) + or: [self deny: (self isInFutureSpace: objOop). + self isReallyYoungObject: objOop]! - or: [self isReallyYoungObject: objOop]! Item was changed: ----- Method: StackInterpreter>>followForwardingPointersInStackZone: (in category 'object memory support') ----- followForwardingPointersInStackZone: theBecomeEffectsFlags "Spur's become: is lazy, turning the becommed object into a forwarding object to the other. The read-barrier is minimised by arranging that forwarding pointers will fail a method cache probe, since notionally objects' internals are accessed only via sending messages to them, the exception is primitives that access the internals of the non-receiver argument(s). To avoid a read barrier on bytecode, literal and inst var fetch we scan the receivers and methods in the stack zone and follow any forwarded ones. This is of course way cheaper than scanning all of memory as in the old become." | theIPPtr | <inline: false> <var: #thePage type: #'StackPage *'> <var: #theSP type: #'char *'> <var: #theFP type: #'char *'> <var: #callerFP type: #'char *'> <var: #theIPPtr type: #usqInt> (theBecomeEffectsFlags anyMask: BecameCompiledMethodFlag) ifTrue: [(objectMemory isForwarded: method) ifTrue: [theIPPtr := instructionPointer - method. method := objectMemory followForwarded: method. instructionPointer := method + theIPPtr]. (objectMemory isForwarded: newMethod) ifTrue: [newMethod := objectMemory followForwarded: newMethod]]. self assert: stackPage ~= 0. 0 to: numStackPages - 1 do: [:i| | thePage theSP theFP callerFP theIP oop | thePage := stackPages stackPageAt: i. thePage isFree ifFalse: [theSP := thePage headSP. theFP := thePage headFP. "Skip the instruction pointer on top of stack of inactive pages." thePage = stackPage ifTrue: [theIPPtr := 0] ifFalse: [theIPPtr := theSP asInteger. theSP := theSP + objectMemory wordSize]. [self assert: (thePage addressIsInPage: theFP). self assert: (theIPPtr = 0 or: [thePage addressIsInPage: theIPPtr asVoidPointer]). oop := stackPages longAt: theFP + FoxReceiver. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theFP + FoxReceiver put: (objectMemory followForwarded: oop)]. theIP := (theFP + (self frameStackedReceiverOffset: theFP)) asInteger. "reuse theIP; its just an offset here" oop := stackPages longAt: theIP. (objectMemory isOopForwarded: oop) ifTrue: [stackPages longAt: theIP put: (objectMemory followForwarded: oop)]. ((self frameHasContext: theFP) and: [(objectMemory isForwarded: (self frameContext: theFP))]) ifTrue: [stackPages longAt: theFP + FoxThisContext put: (objectMemory followForwarded: (self frameContext: theFP))]. oop := self frameMethod: theFP. (objectMemory isForwarded: oop) ifTrue: [| newOop delta | newOop := objectMemory followForwarded: oop. theIPPtr ~= 0 ifTrue: [self assert: (stackPages longAt: theIPPtr) > (self frameMethod: theFP). delta := newOop - oop. stackPages longAt: theIPPtr put: (stackPages longAt: theIPPtr) + delta]. stackPages longAt: theFP + FoxMethod put: (oop := newOop)]. (callerFP := self frameCallerFP: theFP) ~= 0] whileTrue: [theIPPtr := (theFP + FoxCallerSavedIP) asInteger. + theFP := callerFP]. + theSP := theFP + FoxCallerContext. "a.k.a. FoxCallerSavedIP" + [theSP <= thePage baseAddress] whileTrue: + [oop := stackPages longAt: theSP. + (objectMemory isForwarded: oop) ifTrue: + [stackPages longAt: theSP put: (objectMemory followForwarded: oop)]. + theSP := theSP + objectMemory wordSize]]]! - theFP := callerFP]]]! Item was changed: ----- Method: StackInterpreter>>makeBaseFrameFor: (in category 'frame access') ----- makeBaseFrameFor: aContext "<Integer>" "Marry aContext with the base frame of a new stack page. Build the base frame to reflect the context's state. Answer the new page." <returnTypeC: #'StackPage *'> | page pointer theMethod theIP numArgs stackPtrIndex maybeClosure rcvr | <inline: false> <var: #page type: #'StackPage *'> <var: #pointer type: #'char *'> + self assert: (objectMemory isContext: aContext). self assert: (self isSingleContext: aContext). self assert: (objectMemory goodContextSize: aContext). page := self newStackPage. pointer := page baseAddress. theIP := objectMemory fetchPointer: InstructionPointerIndex ofObject: aContext. theMethod := objectMemory followObjField: MethodIndex ofObject: aContext. (objectMemory isIntegerObject: theIP) ifFalse: [self error: 'context is not resumable']. theIP := objectMemory integerValueOf: theIP. rcvr := objectMemory followField: ReceiverIndex ofObject: aContext. "If the frame is a closure activation then the closure should be on the stack in the pushed receiver position (closures receiver the value[:value:] messages). Otherwise it should be the receiver proper." maybeClosure := objectMemory followField: ClosureIndex ofObject: aContext. maybeClosure ~= objectMemory nilObject ifTrue: [numArgs := self argumentCountOfClosure: maybeClosure. stackPages longAt: pointer put: maybeClosure] ifFalse: [| header | header := objectMemory methodHeaderOf: theMethod. numArgs := self argumentCountOfMethodHeader: header. "If this is a synthetic context its IP could be pointing at the CallPrimitive opcode. If so, skip it." (theIP signedIntFromLong > 0 and: [(self methodHeaderHasPrimitive: header) and: [theIP = (1 + (objectMemory lastPointerOf: theMethod))]]) ifTrue: [theIP := theIP + (self sizeOfCallPrimitiveBytecode: header)]. stackPages longAt: pointer put: rcvr]. "Put the arguments on the stack" 1 to: numArgs do: [:i| stackPages longAt: (pointer := pointer - objectMemory wordSize) put: (objectMemory fetchPointer: ReceiverIndex + i ofObject: aContext)]. "saved caller ip is sender context in base frame" stackPages longAt: (pointer := pointer - objectMemory wordSize) put: (objectMemory fetchPointer: SenderIndex ofObject: aContext). "base frame's saved fp is null" stackPages longAt: (pointer := pointer - objectMemory wordSize) put: 0. page baseFP: pointer; headFP: pointer. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: theMethod. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: (self encodeFrameFieldHasContext: true isBlock: maybeClosure ~= objectMemory nilObject numArgs: numArgs). self assert: (self frameHasContext: page baseFP). self assert: (self frameNumArgs: page baseFP) == numArgs. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: aContext. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: rcvr. stackPtrIndex := self quickFetchInteger: StackPointerIndex ofObject: aContext. self assert: ReceiverIndex + stackPtrIndex < (objectMemory lengthOf: aContext). numArgs + 1 to: stackPtrIndex do: [:i| stackPages longAt: (pointer := pointer - objectMemory wordSize) put: (objectMemory fetchPointer: ReceiverIndex + i ofObject: aContext)]. "top of stack is the instruction pointer" theIP := self iframeInstructionPointerForIndex: theIP method: theMethod. stackPages longAt: (pointer := pointer - objectMemory wordSize) put: theIP. page headSP: pointer. self assert: (self context: aContext hasValidInversePCMappingOf: theIP in: page baseFP). "Mark context as married by setting its sender to the frame pointer plus SmallInteger tags and the InstructionPointer to the saved fp (which ensures correct alignment w.r.t. the frame when we check for validity) plus SmallInteger tags." objectMemory storePointerUnchecked: SenderIndex ofObject: aContext withValue: (self withSmallIntegerTags: page baseFP). objectMemory storePointerUnchecked: InstructionPointerIndex ofObject: aContext withValue: (self withSmallIntegerTags: 0). self assert: (objectMemory isIntegerObject: (objectMemory fetchPointer: SenderIndex ofObject: aContext)). self assert: (self frameOfMarriedContext: aContext) = page baseFP. self assert: (self validStackPageBaseFrame: page). ^page! Item was changed: ----- Method: StackInterpreter>>mapStackPages (in category 'object memory support') ----- mapStackPages + <inline: #never> - <inline: false> <var: #thePage type: #'StackPage *'> <var: #theSP type: #'char *'> <var: #theFP type: #'char *'> <var: #callerFP type: #'char *'> <var: #theIPPtr type: #'char *'> 0 to: numStackPages - 1 do: [:i| | thePage theSP theFP callerFP theIPPtr theIP oop | thePage := stackPages stackPageAt: i. thePage isFree ifFalse: [theSP := thePage headSP. theFP := thePage headFP. "Skip the instruction pointer on top of stack of inactive pages." thePage = stackPage ifTrue: [theIPPtr := 0] ifFalse: [theIPPtr := theSP. theSP := theSP + objectMemory wordSize]. [self assert: (thePage addressIsInPage: theFP). self assert: (thePage addressIsInPage: theSP). self assert: (theIPPtr = 0 or: [thePage addressIsInPage: theIPPtr]). [theSP <= (theFP + FoxReceiver)] whileTrue: [oop := stackPages longAt: theSP. (objectMemory shouldRemapOop: oop) ifTrue: [stackPages longAt: theSP put: (objectMemory remapObj: oop)]. theSP := theSP + objectMemory wordSize]. (self frameHasContext: theFP) ifTrue: [(objectMemory shouldRemapObj: (self frameContext: theFP)) ifTrue: [stackPages longAt: theFP + FoxThisContext put: (objectMemory remapObj: (self frameContext: theFP))]. "With SqueakV3 objectMemory can't assert since object body is yet to move." objectMemory hasSpurMemoryManagerAPI ifTrue: [self assert: ((self isMarriedOrWidowedContext: (self frameContext: theFP)) and: [(self frameOfMarriedContext: (self frameContext: theFP)) = theFP])]]. (objectMemory shouldRemapObj: (self frameMethod: theFP)) ifTrue: [theIPPtr ~= 0 ifTrue: [self assert: (stackPages longAt: theIPPtr) > (self frameMethod: theFP). theIP := (stackPages longAt: theIPPtr) - (self frameMethod: theFP)]. stackPages longAt: theFP + FoxMethod put: (objectMemory remapObj: (self frameMethod: theFP)). theIPPtr ~= 0 ifTrue: [stackPages longAt: theIPPtr put: theIP + (self frameMethod: theFP)]]. (callerFP := self frameCallerFP: theFP) ~= 0] whileTrue: [theSP := (theIPPtr := theFP + FoxCallerSavedIP) + objectMemory wordSize. theFP := callerFP]. theSP := theFP + FoxCallerContext. "a.k.a. FoxCallerSavedIP" [theSP <= thePage baseAddress] whileTrue: [oop := stackPages longAt: theSP. (objectMemory shouldRemapOop: oop) ifTrue: [stackPages longAt: theSP put: (objectMemory remapObj: oop)]. theSP := theSP + objectMemory wordSize]]]! Item was changed: ----- Method: StackInterpreter>>withSmallIntegerTags: (in category 'frame access') ----- withSmallIntegerTags: value <inline: true> <var: #value type: #'char *'> self assert: ((self oopForPointer: value) bitAnd: objectMemory wordSize - 1) = 0. + ^(self + cCode: [self oopForPointer: value] + inSmalltalk: [value bitAnd: objectMemory maxCInteger]) + objectMemory smallIntegerTag! - ^(self oopForPointer: value) + 1! Item was removed: - ----- Method: StackInterpreterSimulator>>withSmallIntegerTags: (in category 'frame access') ----- - withSmallIntegerTags: value - "The simulator works with strictly positive bit patterns" - ^value < 0 - ifTrue: [(value bitAnd: 16rFFFFFFFF) + 1] - ifFalse: [value + 1]!

1 0

VM Maker: VMMaker.oscog-eem.1011.mcz
by commits＠source.squeak.org 09 Jan '15

09 Jan '15

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.1011.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.1011 Author: eem Time: 8 January 2015, 1:01:11.779 pm UUID: dd0dbcf0-1b1e-4d40-854c-130590e38268 Ancestors: VMMaker.oscog-eem.1010 Spur: Fix bad bug introduced in become changes in VMMaker.oscog-eem.841 through 844. The class table contains puns, e.g. using Array and WeakArray. These class index puns must not mislead the allInstances primitive into concluding that the classes in question are at multiple indices and hence avoid wrongly purging them. =============== Diff against VMMaker.oscog-eem.1010 =============== Item was changed: ----- Method: SpurMemoryManager>>enterIntoClassTable: (in category 'class table') ----- enterIntoClassTable: aBehavior "Enter aBehavior into the class table and answer 0. Otherwise answer a primitive failure code." <inline: false> | initialMajorIndex majorIndex minorIndex page | majorIndex := classTableIndex >> self classTableMajorIndexShift. initialMajorIndex := majorIndex. "classTableIndex should never index the first page; it's reserved for known classes" self assert: initialMajorIndex > 0. minorIndex := classTableIndex bitAnd: self classTableMinorIndexMask. [page := self fetchPointer: majorIndex ofObject: hiddenRootsObj. page = nilObj ifTrue: [page := self allocateSlotsInOldSpace: self classTablePageSize format: self arrayFormat classIndex: self arrayClassIndexPun. page ifNil: [^PrimErrNoMemory]. self fillObj: page numSlots: self classTablePageSize with: nilObj. self storePointer: majorIndex ofObject: hiddenRootsObj withValue: page. numClassTablePages := numClassTablePages + 1. minorIndex := 0]. minorIndex to: self classTablePageSize - 1 do: [:i| (self fetchPointer: i ofObject: page) = nilObj ifTrue: + [classTableIndex := majorIndex << self classTableMajorIndexShift + i.. + "classTableIndex must never index the first page, which is reserved for classes known to the VM." + self assert: classTableIndex >= (1 << self classTableMajorIndexShift). - [classTableIndex := majorIndex << self classTableMajorIndexShift + i. self storePointer: i ofObject: page withValue: aBehavior. self setHashBitsOf: aBehavior to: classTableIndex. self assert: (self classAtIndex: (self rawHashBitsOf: aBehavior)) = aBehavior. ^0]]. majorIndex := (majorIndex + 1 bitAnd: self classIndexMask) max: 1. majorIndex = initialMajorIndex ifTrue: "wrapped; table full" [^PrimErrLimitExceeded]] repeat! Item was changed: ----- Method: SpurMemoryManager>>expungeDuplicateAndUnmarkedClasses: (in category 'class table') ----- expungeDuplicateAndUnmarkedClasses: expungeUnmarked "Bits have been set in the classTableBitmap corresponding to used classes. Any class in the class table that does not have a bit set has no instances with that class index. However, becomeForward: can create duplicate entries, and these duplicate entries wont match their identityHash. So expunge duplicates by eliminating unmarked entries that don't occur at their identityHash." 1 to: numClassTablePages - 1 do: "Avoid expunging the puns by not scanning the 0th page." [:i| | classTablePage | classTablePage := self fetchPointer: i ofObject: hiddenRootsObj. 0 to: self classTablePageSize - 1 do: [:j| | classOrNil classIndex | classOrNil := self fetchPointer: j ofObject: classTablePage. classIndex := i << self classTableMajorIndexShift + j. self assert: (classOrNil = nilObj or: [coInterpreter addressCouldBeClassObj: classOrNil]). "only remove a class if it is at a duplicate entry or it is unmarked and we're expunging unmarked classes." classOrNil = nilObj ifTrue: [classIndex < classTableIndex ifTrue: [classTableIndex := classIndex]] ifFalse: [((expungeUnmarked and: [(self isMarked: classOrNil) not]) or: [(self rawHashBitsOf: classOrNil) ~= classIndex]) ifTrue: [self storePointerUnchecked: j ofObject: classTablePage withValue: nilObj. "but if it is marked, it should still be in the table at its correct index." self assert: ((expungeUnmarked and: [(self isMarked: classOrNil) not]) or: [(self classAtIndex: (self rawHashBitsOf: classOrNil)) = classOrNil]). "If the removed class is before the classTableIndex, set the classTableIndex to point to the empty slot so as to reuse it asap." classIndex < classTableIndex ifTrue: + [classTableIndex := classIndex]]]]]. + "classTableIndex must never index the first page, which is reserved for classes known to the VM." + self assert: classTableIndex >= (1 << self classTableMajorIndexShift)! - [classTableIndex := classIndex]]]]]! Item was changed: ----- Method: SpurMemoryManager>>expungeFromClassTable: (in category 'class table') ----- expungeFromClassTable: aBehavior "Remove aBehavior from the class table." <inline: false> | classIndex majorIndex minorIndex classTablePage | self assert: (self isInClassTable: aBehavior). classIndex := self rawHashBitsOf: aBehavior. majorIndex := classIndex >> self classTableMajorIndexShift. minorIndex := classIndex bitAnd: self classTableMinorIndexMask. classTablePage := self fetchPointer: majorIndex ofObject: hiddenRootsObj. self assert: classTablePage ~= classTableFirstPage. self assert: (self numSlotsOf: classTablePage) = self classTablePageSize. self assert: (self fetchPointer: minorIndex ofObject: classTablePage) = aBehavior. self storePointerUnchecked: minorIndex ofObject: classTablePage withValue: nilObj. "If the removed class is before the classTableIndex, set the classTableIndex to point to the empty slot so as to reuse it asap." classIndex < classTableIndex ifTrue: + [classTableIndex := classIndex]. + "classTableIndex must never index the first page, which is reserved for classes known to the VM." + self assert: classTableIndex >= (1 << self classTableMajorIndexShift)! - [classTableIndex := classIndex]! Item was changed: ----- Method: SpurMemoryManager>>isClassAtUniqueIndex: (in category 'class table') ----- isClassAtUniqueIndex: aClass + "Answer if aClass exists at only one index in the class table. Be careful not to + be misled by classes that have puns, such as Array." - "Answer if aClass exists at only one index in the class table." | expectedIndex | expectedIndex := self rawHashBitsOf: aClass. self classTableEntriesDo: [:entry :index| + (entry = aClass + and: [index ~= expectedIndex + and: [index > self lastClassIndexPun]]) ifTrue: - (entry = aClass and: [index ~= expectedIndex]) ifTrue: [^false]]. ^true! Item was changed: ----- Method: SpurMemoryManager>>postBecomeScanClassTable: (in category 'become implementation') ----- postBecomeScanClassTable: effectsFlags "Scan the class table post-become (iff an active class object was becommed) to ensure no forwarding pointers, and no unhashed classes exist in the class table. Note that one-way become can cause duplications in the class table. When can these be eliminated? We use the classTableBitmap to mark classTable entries (not the classes themselves, since marking a class doesn't help in knowing if its index is used). On image load, and during incrememtal scan-mark and full GC, classIndices are marked. We can somehow avoid following classes from the classTable until after this mark phase." self assert: self validClassTableRootPages. (effectsFlags anyMask: BecamePointerObjectFlag) ifFalse: [^self]. 0 to: numClassTablePages - 1 do: [:i| | page | page := self fetchPointer: i ofObject: hiddenRootsObj. self assert: (self isForwarded: page) not. 0 to: (self numSlotsOf: page) - 1 do: [:j| | classOrNil | classOrNil := self fetchPointer: j ofObject: page. classOrNil ~= nilObj ifTrue: [(self isForwarded: classOrNil) ifTrue: [classOrNil := self followForwarded: classOrNil. self storePointer: j ofObject: page withValue: classOrNil]. (self rawHashBitsOf: classOrNil) = 0 ifTrue: [self storePointerUnchecked: j ofObject: page withValue: nilObj. "If the removed class is before the classTableIndex, set the classTableIndex to point to the empty slot so as to reuse it asap." (i << self classTableMajorIndexShift + j) < classTableIndex ifTrue: + [classTableIndex := i << self classTableMajorIndexShift + j]]]]]. + "classTableIndex must never index the first page, which is reserved for classes known to the VM." + self assert: classTableIndex >= (1 << self classTableMajorIndexShift)! - [classTableIndex := i << self classTableMajorIndexShift + j]]]]]! Item was changed: ----- Method: SpurMemoryManager>>purgeDuplicateClassTableEntriesFor: (in category 'class table') ----- purgeDuplicateClassTableEntriesFor: aClass "Given that either marking or allnstances has ensured that all instances of aClass have the class's hash as their class index, ensure aClass is in the table only at its hash." | expectedIndex | expectedIndex := self rawHashBitsOf: aClass. self classTableEntriesDo: [:entry :index| (entry = aClass and: [index ~= expectedIndex]) ifTrue: [self classAtIndex: index put: nilObj. index < classTableIndex ifTrue: + [classTableIndex := index]]]. + "classTableIndex must never index the first page, which is reserved for classes known to the VM." + self assert: classTableIndex >= (1 << self classTableMajorIndexShift)! - [classTableIndex := index]]]!

1 0

VM Maker: VMMaker.oscog-eem.1010.mcz
by commits＠source.squeak.org 09 Jan '15

09 Jan '15

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.1010.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.1010 Author: eem Time: 6 January 2015, 10:52:02.447 am UUID: 9065e685-b193-428f-9c7a-27ce1afe8c71 Ancestors: VMMaker.oscog-eem.1009 Be lenient w.r.t. unpinning married contexts. Move the check down into StackInterpreter's version. This allows e.g. self allObjectsDo: [:o| o unpin] =============== Diff against VMMaker.oscog-eem.1009 =============== Item was changed: ----- Method: InterpreterPrimitives>>primitivePin (in category 'memory space primitives') ----- primitivePin "Pin or unpin the receiver, i.e. make it immobile or mobile, based on the argument. Answer whether the object was already pinned. N.B. pinning does *not* prevent an object from being garbage collected." | obj boolean wasPinned | objectMemory hasSpurMemoryManagerAPI ifFalse: [^self primitiveFailFor: PrimErrUnsupported]. obj := self stackValue: 1. ((objectMemory isImmediate: obj) + or: [objectMemory isForwarded: obj]) ifTrue: - or: [(objectMemory isForwarded: obj) - or: [(objectMemory isContext: obj) - and: [self isStillMarriedContext: obj]]]) ifTrue: [^self primitiveFailFor: PrimErrBadReceiver]. + boolean := self stackTop. (boolean = objectMemory falseObject or: [boolean = objectMemory trueObject]) ifFalse: [^self primitiveFailFor: PrimErrBadArgument]. (objectMemory isPinned: obj) ifTrue: [wasPinned := objectMemory trueObject. + boolean ~= wasPinned ifTrue: + [objectMemory setIsPinnedOf: obj to: false]] - objectMemory setIsPinnedOf: obj to: boolean = objectMemory trueObject] ifFalse: [wasPinned := objectMemory falseObject. (boolean = objectMemory trueObject and: [(objectMemory pinObject: obj) = 0]) ifTrue: [^self primitiveFailFor: PrimErrNoMemory]]. self pop: argumentCount + 1 thenPush: wasPinned! Item was added: + ----- Method: StackInterpreterPrimitives>>primitivePin (in category 'memory space primitives') ----- + primitivePin + "Pin or unpin the receiver, i.e. make it immobile or mobile, based on the argument. + Answer whether the object was already pinned. N.B. pinning does *not* prevent + an object from being garbage collected." + | obj boolean wasPinned | + objectMemory hasSpurMemoryManagerAPI ifFalse: + [^self primitiveFailFor: PrimErrUnsupported]. + + obj := self stackValue: 1. + ((objectMemory isImmediate: obj) + or: [objectMemory isForwarded: obj]) ifTrue: + [^self primitiveFailFor: PrimErrBadReceiver]. + + boolean := self stackTop. + (boolean = objectMemory falseObject + or: [boolean = objectMemory trueObject]) ifFalse: + [^self primitiveFailFor: PrimErrBadArgument]. + + (objectMemory isPinned: obj) + ifTrue: + [wasPinned := objectMemory trueObject. + boolean ~= wasPinned ifTrue: + [objectMemory setIsPinnedOf: obj to: false]] + ifFalse: + [wasPinned := objectMemory falseObject. + boolean = objectMemory trueObject ifTrue: + [((objectMemory isContext: obj) + and: [self isStillMarriedContext: obj]) ifTrue: + [^self primitiveFailFor: PrimErrBadReceiver]. + (objectMemory pinObject: obj) = 0 ifTrue: + [^self primitiveFailFor: PrimErrNoMemory]]]. + + self pop: argumentCount + 1 thenPush: wasPinned!

1 0

VM Maker: VMMaker.oscog-eem.1009.mcz
by commits＠source.squeak.org 09 Jan '15

09 Jan '15

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.1009.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.1009 Author: eem Time: 3 January 2015, 12:26:13.686 pm UUID: b7476b8c-cba9-42a5-8b0d-676775d111ef Ancestors: VMMaker.oscog-eem.1008 Fix slip in primitivePin. =============== Diff against VMMaker.oscog-eem.1008 =============== Item was changed: ----- Method: InterpreterPrimitives>>primitivePin (in category 'memory space primitives') ----- primitivePin "Pin or unpin the receiver, i.e. make it immobile or mobile, based on the argument. Answer whether the object was already pinned. N.B. pinning does *not* prevent an object from being garbage collected." | obj boolean wasPinned | objectMemory hasSpurMemoryManagerAPI ifFalse: [^self primitiveFailFor: PrimErrUnsupported]. obj := self stackValue: 1. ((objectMemory isImmediate: obj) or: [(objectMemory isForwarded: obj) or: [(objectMemory isContext: obj) and: [self isStillMarriedContext: obj]]]) ifTrue: [^self primitiveFailFor: PrimErrBadReceiver]. boolean := self stackTop. (boolean = objectMemory falseObject or: [boolean = objectMemory trueObject]) ifFalse: [^self primitiveFailFor: PrimErrBadArgument]. (objectMemory isPinned: obj) ifTrue: [wasPinned := objectMemory trueObject. objectMemory setIsPinnedOf: obj to: boolean = objectMemory trueObject] ifFalse: [wasPinned := objectMemory falseObject. (boolean = objectMemory trueObject + and: [(objectMemory pinObject: obj) = 0]) ifTrue: - and: [objectMemory pinObject: obj]) = 0 ifTrue: [^self primitiveFailFor: PrimErrNoMemory]]. self pop: argumentCount + 1 thenPush: wasPinned!

1 0

diff of changes to http://www.squeakvm.org/svn/squeak/branches/Cog/image build scripts
by Paul DeBruicker 09 Jan '15

09 Jan '15

Hi - I'm building a VMMaker image to fix an image I've got where I put a #halt in a method thats used during startup. To do that I had to make changes to the envvars.sh and getGoodCogVM.sh files. I've attached a diff. The Cog VM those scripts use is #3104. Just bringing that up because the most recent is 3205. Hope this helps Paul

3 2

Cannot commit VMMaker to source.squeak.org
by Eliot Miranda 09 Jan '15

09 Jan '15

Hi Levente, not sure if this is due to the replacement of box2 (I suspect it is) but in trying to upload the VMMaker.oscog versions I've been keeping back while box2 is repaired/replaced I get the following error: HTTP/1.1 413 Request Entity Too Large Server: nginx Date: Fri, 09 Jan 2015 01:00:40 GMT Content-Type: text/html Content-Length: 192 Connection: closeHTTP/1.1 413 Request Entity Too Large Server: nginx Date: Fri, 09 Jan 2015 01:00:40 GMT Content-Type: text/html Content-Length: 192 Connection: close<html> <head><title>413 Request Entity Too Large</title></head> <body bgcolor="white"> <center><h1>413 Request Entity Too Large</h1></center> <hr><center>nginx</center> </body> </html> The sizes of the packages are currently McStalker.Cog$ ls -lh package-cache/VMMaker.oscog-eem.101* -rw-r--r--@ 1 eliot staff 3.5M Jan 6 10:52 package-cache/VMMaker.oscog-eem.1010.mcz -rw-r--r--@ 1 eliot staff 3.5M Jan 8 13:01 package-cache/VMMaker.oscog-eem.1011.mcz -rw-r--r--@ 1 eliot staff 3.5M Jan 8 17:00 package-cache/VMMaker.oscog-eem.1012.mcz I think we should allow at least 5m. Is this possible or will I have to split up VMMaker?? (oh no, Mr Billllll!) -- best, Eliot

3 3

[squeak-dev] Re: [Vm-dev] [OSProcess] forking and file descriptors
by Max Leske 09 Jan '15

09 Jan '15

(Resending with proper subject…) > On 08 Jan 2015, at 20:48, squeak-dev-request(a)lists.squeakfoundation.org <mailto:squeak-dev-request@lists.squeakfoundation.org> wrote: > > Date: Thu, 8 Jan 2015 16:56:30 +0100 > From: Henrik Johansen <henrik.s.johansen(a)veloxit.no <mailto:henrik.s.johansen@veloxit.no>> > Subject: [squeak-dev] Re: [Vm-dev] [OSProcess] forking and file > descriptors > > >> On 08 Jan 2015, at 11:37 , Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >> >> >> Hi >> >> We currently use ImageSegment to create snapshots of our object graphs. To ensure consistency (and for performance reasons) we create a fork of the image and then run the segment creation in the fork. We’ve always had minor issues with TCP sockets but they are pretty rare and have never corrupted any data (we close the TCP connections in the child). >> >> Recently however, we created a new application which also makes heavy use of a database and now it seems that forking creates a real problem. In anticipation of possible problems I opted to destroy all sockets (with Socket>>destroy) in the fork, thinking that, since all file descriptors are copies of the ones in the parent process, the sockets in the parent process should be unaffected [1], [2]. >> With that mechanism in place however, we are seeing very weird things, such as multiples sockets in the parent (!) having the same file handle (which leads to the wrong data being read from the database and, in turn, corrupt objects). >> >> AFAICT, the OSProcess plugin doesn’t offer any way of dealing with such problems so I was wondering if anybody has had any experience with these kinds of issues and whether there is some kind of best practice. >> >> I am aware that the most simple option is to close the sockets in the parent before forking, but that will mean that we would have to wait for all database connections to finish executing, then blocking them to prevent new connections to the database. Depending on the time a query takes (which may well be a couple of seconds in our case) clients would need to wait for quite a long time before their request can be answered (and this scenario of course assumes that we only close the database sockets and leave the TCP sockets open…). >> >> So under the condition that I need to fork that image, what is the best way to deal with open file descriptors? >> >> >> Thanks for your time. >> Max >> >> [1] http://man7.org/linux/man-pages/man2/fork.2.html <http://man7.org/linux/man-pages/man2/fork.2.html> >> [2] http://man7.org/linux/man-pages/man2/clone.2.html <http://man7.org/linux/man-pages/man2/clone.2.html> > > Well... > If I understand the source correctly (at least on Unix, https://github.com/pharo-project/pharo-vm/blob/master/platforms/unix/plugin… <https://github.com/pharo-project/pharo-vm/blob/master/platforms/unix/plugin…> <https://github.com/pharo-project/pharo-vm/blob/master/platforms/unix/plugin… <https://github.com/pharo-project/pharo-vm/blob/master/platforms/unix/plugin…>>) > The socketHandle in a Socket instance is a pointer to a private (platform-specific) struct. > That struct again has a handle to the native socket, which I assume is what gets copied when you fork a process? > > Socket >> primDestroySocket frees the memory pointed to by socketHandle. Hm… that gives me an idea: assume that everything works as advertised and that the child process gets copies of the socket descriptors (which can be closed safely without intefering with the parent). If I’m right, the Socket instances in the image hold on to the address of the *parent* handle (in an inst var). So now, when I close a socket with #primSocketDestroy:, the handle passed to the plugin will be the handle of the parent socket (although it sounds strange that the child should be able to close a file descriptor of its parent…). That would mean that I must not close any sockets in the child. One option, it seems to me, is to suspend all processes that use sockets. Terminating them might pose another problem, if socket destruction is part of an unwind block in one of the processes (e.g. TCP connections in Seaside) then sockets will be destroyed during termination. Another option: set all the socket handles to nil, then terminate the processes (yes ugly, but it might just work…). > So, are you using clone or fork to create a fork of the image? OSProcess uses plain fork() (in forkSqueak()). That’s what I use from the image. > If their memory is shared (clone) instead of copied (fork), you might be kicking the feet out from under the parent image as well, so to speak… From my understanding the file descriptors should be copied (fork). So that shouldn’t happen (but see above…). Thanks Henry. > > Cheers, > Henry

3 2

Re: [squeak-dev] [Vm-dev] [OSProcess] forking and file descriptors
by Max Leske 09 Jan '15

09 Jan '15

> On 09 Jan 2015, at 10:55, squeak-dev-request(a)lists.squeakfoundation.org wrote: > > Date: Fri, 9 Jan 2015 10:55:22 +0100 > From: Henrik Johansen <henrik.s.johansen(a)veloxit.no <mailto:henrik.s.johansen@veloxit.no>> > Subject: Re: [squeak-dev] Re: [Vm-dev] [OSProcess] forking and file > descriptors > >> On 09 Jan 2015, at 10:44 , Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >> >> >>> On 09 Jan 2015, at 10:25, vm-dev-request(a)lists.squeakfoundation.org <mailto:vm-dev-request@lists.squeakfoundation.org> <mailto:vm-dev-request@lists.squeakfoundation.org <mailto:vm-dev-request@lists.squeakfoundation.org>> wrote: >>> >>> Date: Fri, 9 Jan 2015 10:24:42 +0100 >>> From: Henrik Johansen <henrik.s.johansen(a)veloxit.no <mailto:henrik.s.johansen@veloxit.no> <mailto:henrik.s.johansen@veloxit.no <mailto:henrik.s.johansen@veloxit.no>>> >>> Subject: Re: [squeak-dev] [Vm-dev] [OSProcess] forking and file >>> descriptors >>> >>> >>>> On 09 Jan 2015, at 9:32 , Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com> <mailto:maxleske@gmail.com <mailto:maxleske@gmail.com>>> wrote: >>>> >>>> >>> >>>> That would mean that I must not close any sockets in the child. One option, it seems to me, is to suspend all processes that use sockets. Terminating them might pose another problem, if socket destruction is part of an unwind block in one of the processes (e.g. TCP connections in Seaside) then sockets will be destroyed during termination. >>>> >>>> Another option: set all the socket handles to nil, then terminate the processes (yes ugly, but it might just work…). >>> >>> Just beware you might run into the issue that resuming a processes waiting for a semaphore will proceed as if the semaphore were signalled, >>> Can't tell offhand if that would actually be a problem in this case, or if the affected processes would promptly resume waiting after socket read/writes may initiate with no data. >> >> I wouldn’t resume any of those processes if I can help it. After creating the segment the child kills itself. Thanks for pointing that out though. > > I was thinking you'd need to suspend the processes in the parent, before forking off a child, (to ensure there will be no possibility of a process running before it's suspended in the child) then resume after forking? I’m hoping that I can get around that by using #valueUninterruptibly (or similar). It would be much nicer to leave the parent as it is and do everything related to the snapshot in the child. But maybe I’ll have to suspend the processes in the parent in the end. > >> >>> >>> Another (probably non-portable, which would be painful if not consistent across platforms) option: >>> Forget about image-side handling, and alter the SocketPlugin to set FD_CLOEXEC if available when opening sockets. (It's in http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/fcntl.h.html <http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/fcntl.h.html><http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/fcntl.h.html <http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/fcntl.h.html>> , but that's rather new) >>> On newer Linuxen, you also have SOCK_CLOEXEC for socket(), which opens/sets in an atomic operation, but the race condition avoided by that is hardly relevant in our case. >> >> True. But I really don’t want to maintain a branch of OSProcess :) > It wouldn't be a branch of OSProcess, but an update to the platform SocketPlugin sources. > > I was probably wrong and it's not so new (wouldn't it be nice if all API's had the same structure as Postgres, where checking previous versions is *really* easy?), the 2004 version of the standard also includes it. A cursory check indicates at least OSX/*BSD/*Solaris would support it as well. > Doing the equivalent on Windows seems to have certain caveats though... > http://stackoverflow.com/questions/12058911/can-tcp-socket-handles-be-set-n… <http://stackoverflow.com/questions/12058911/can-tcp-socket-handles-be-set-n…> I’m not entirely sure that what you are suggesting will work, at least not with FD_CLOEXEC. man fcntl says: File descriptor flags The following commands manipulate the flags associated with a file descriptor. Currently, only one such flag is defined: FD_CLOEXEC , the close-on-exec flag. If the FD_CLOEXEC bit is 0, the file descriptor will remain open across an execve(2) , otherwise it will be closed. IIRC fork doesn’t use exec, so this flag doesn’t change anything for my scenario. [1] seems to confirm this. Cheers, Max [1] http://stackoverflow.com/questions/5713242/linux-fork-prevent-file-descript… > > Cheers, > Henry

1 0

[squeak-dev] Re: [Vm-dev] [OSProcess] forking and file descriptors
by Max Leske 09 Jan '15

09 Jan '15

> On 09 Jan 2015, at 10:25, vm-dev-request(a)lists.squeakfoundation.org wrote: > > Date: Fri, 9 Jan 2015 10:24:42 +0100 > From: Henrik Johansen <henrik.s.johansen(a)veloxit.no <mailto:henrik.s.johansen@veloxit.no>> > Subject: Re: [squeak-dev] [Vm-dev] [OSProcess] forking and file > descriptors > > >> On 09 Jan 2015, at 9:32 , Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >> >> > >> That would mean that I must not close any sockets in the child. One option, it seems to me, is to suspend all processes that use sockets. Terminating them might pose another problem, if socket destruction is part of an unwind block in one of the processes (e.g. TCP connections in Seaside) then sockets will be destroyed during termination. >> >> Another option: set all the socket handles to nil, then terminate the processes (yes ugly, but it might just work…). > > Just beware you might run into the issue that resuming a processes waiting for a semaphore will proceed as if the semaphore were signalled, > Can't tell offhand if that would actually be a problem in this case, or if the affected processes would promptly resume waiting after socket read/writes may initiate with no data. I wouldn’t resume any of those processes if I can help it. After creating the segment the child kills itself. Thanks for pointing that out though. > > Another (probably non-portable, which would be painful if not consistent across platforms) option: > Forget about image-side handling, and alter the SocketPlugin to set FD_CLOEXEC if available when opening sockets. (It's in http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/fcntl.h.html <http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/fcntl.h.html> , but that's rather new) > On newer Linuxen, you also have SOCK_CLOEXEC for socket(), which opens/sets in an atomic operation, but the race condition avoided by that is hardly relevant in our case. True. But I really don’t want to maintain a branch of OSProcess :) > > Cheers, > Henry

2 1

← Newer
1
...
5
6
7
8
9
10
11
12
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev January 2015