Vm-dev October 2023

vm-dev@lists.squeakfoundation.org

7 participants
17 discussions

VM Maker: VMMaker.threaded-LM.3342.mcz
by commits＠source.squeak.org 10 Oct '23

10 Oct '23

Leon Matthes uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.threaded-LM.3342.mcz ==================== Summary ==================== Name: VMMaker.threaded-LM.3342 Author: LM Time: 10 October 2023, 1:38:40.422616 pm UUID: 6129f12c-a2f0-4b43-bfe4-d8e41bebdb55 Ancestors: VMMaker.threaded-LM.3341 Fix multiple issues regarding thread safety 1. vmOwner -1 can be used to lock the VM to a thread that is not a classical "VM Thread" (e.g. the heartbeat thread). 2. maxWaitingPriority is now an atomic. =============== Diff against VMMaker.threaded-LM.3341 =============== Item was changed: CoInterpreterPrimitives subclass: #CoInterpreterMT instanceVariableNames: 'cogThreadManager checkThreadActivation maxWaitingPriority foreignCallbackPriority deferThreadSwitch disowningVMThread disownCount foreignCallbackProcessSlot activeProcessAffined relinquishing processHasThreadAffinity willNotThreadWarnCount' + classVariableNames: 'DisownVMForProcessorRelinquish OwnVMForeignThreadFlag PerThreadRumpCStackSize PrimNumberRelinquishProcessor ProcessUnaffinedOnDisown ReturnToThreadSchedulingLoop VMAlreadyOwnedHenceDoNotDisown' - classVariableNames: 'DisownFlagsShift DisownVMForProcessorRelinquish OwnVMForeignThreadFlag PerThreadRumpCStackSize PrimNumberRelinquishProcessor ProcessUnaffinedOnDisown ReturnToThreadSchedulingLoop VMAlreadyOwnedHenceDoNotDisown' poolDictionaries: 'VMThreadingConstants' category: 'VMMaker-Multithreading'! Item was changed: ----- Method: CoInterpreterMT class>>initializeSchedulerIndices (in category 'initialization') ----- initializeSchedulerIndices super initializeSchedulerIndices. "Class Process" "The thread id of a process is either nil or a SmallInteger that defines how a process binds to threads. If nil, the process may run on any thread. The least significant bit of threadId is a flag. The most significant bits are a threadId. If the threadId is nil the process can run on any thread. If the flag (least significant bit) is set then If the threadId is positive the process can only be run on the thread with that thread Id. If the threadId is negative the process must not be run on the thread with that thread Id. If the flag (least significant bit) is not set then the thread id will not be negative and if non-zero is the id of the thread the process is currenty running on The flag is probably a mistake..." "In part, what's really going on here is an attempt to deal with threading on Mac. Events can only be delivered on the GUi thread. To avoid blocking and/or crashing the VM by making a blocking (e.g. FFI) call on the GUI thread we want a nice way of preventing a process from running on the GUI thread. We can then use a process whose threadId precludes running on the GUI thread to make blocking calls. The alternative, of arranging that events are delivered to a thread the VM does not run on, is problematic; it cannot support event callbacks." "So if we simplify, and threadId is only instructive to the VM, (i.e. can say ''run on a given thread'', or ''don't run on a given thread'', and ''don't care; run on any thread'') and does not necessarily hold the threadId of the thread the process is currenty bound to, how do we locate the threadId for a process temporarily affined to a thread for the duration of an FFI call? Note that a process *must* be bound to a thread for the duration of a threaded call (or foreign callback) so that return occurs on the correct thread. We can use the least significant bit to mean ''temporarily affined'', but we need to support ''don't run on this thread''. We could use bit fields (yuck); we could allocate a two field object and assign it to threadId when setting a process to not run on a given thread. This isn't so bad; use negative values to mean ''don't run on this thread'', and positive values to mean ''run on this thread''. Split the smallest SmallInteger (32-bit, 1 bit sign, 2-bit tags, leaving 29//2) into two 14 bit fields. The least significant 14 bits are the thread id the receiver is temporarily affined to. The most significant 14 bits are the thread id of the thread the proess is either bound to or excluded from. If zero, the process is agnostic. See CogThreadManager>>#maxNumThreads" ThreadIdIndex := 4. ThreadIdShift := 14. "could be 30 in 64-bits" "disown result/own argument flags" OwnVMForeignThreadFlag := 1. VMAlreadyOwnedHenceDoNotDisown := 2. ProcessUnaffinedOnDisown := 4. "& defined in StackInterpreter are..." DisownVMForFFICall := 16. DisownVMForThreading := 32. "N.B. some of these DisownFlags are replicated in platforms/Cross/vm/sqVirtualMachine.h. Hence they should always be initialized." + DisownVMForProcessorRelinquish := 64.! - DisownVMForProcessorRelinquish := 64. - - DisownFlagsShift := DisownVMForProcessorRelinquish highBit! Item was changed: ----- Method: CoInterpreterMT>>cedeToHigherPriorityThreads (in category 'process primitive support') ----- cedeToHigherPriorityThreads "Invoked from checkForEventsMayContextSwitch: to switch threads if a thread wanting to acquire the VM has higher priority than the active process." + | activeProc processAffinity activeContext activePriority activeThread vmThread waitingPriority | - | activeProc processAffinity activeContext activePriority activeThread vmThread | <var: #activeThread type: #'CogVMThread *'> <var: #vmThread type: #'CogVMThread *'> <inline: false> activeProc := self activeProcess. activePriority := self quickFetchInteger: PriorityIndex ofObject: activeProc. processAffinity := self ownerIndexOfProcess: activeProc. activeThread := cogThreadManager currentVMThread. self assert: (cogThreadManager threadIndex: activeThread index isCompatibleWith: processAffinity). + waitingPriority := self getMaxWaitingPriority. activeThread priority: activePriority. vmThread := cogThreadManager highestPriorityThreadIfHigherThan: activePriority + expectedMax: waitingPriority. + - expectedMax: maxWaitingPriority. (vmThread isNil "no waiting thread of sufficiently high priority. Do not switch." or: [vmThread = activeThread]) "The activeProcess needs to run on a different thread. Leave this to threadSwitchIfNecessary:from: in checkForEventsMayContextSwitch:" ifTrue: + [waitingPriority > activePriority ifTrue: + ["We found no thread of sufficiently high priority, even though waitingPriority indicated there should be one. + So reduce the waiting priority back to the priority of the currently active process." + self reduceWaitingPriorityFrom: waitingPriority to: activePriority]. - [maxWaitingPriority > activePriority ifTrue: - [maxWaitingPriority := activePriority]. ^self]. self assert: vmThread priority > activePriority. self assert: vmThread ~= cogThreadManager currentVMThread. self assertValidExecutionPointe: instructionPointer r: framePointer s: stackPointer. + waitingPriority > vmThread priority ifTrue: + [self reduceWaitingPriorityFrom: waitingPriority to: vmThread priority]. - maxWaitingPriority > vmThread priority ifTrue: - [maxWaitingPriority := vmThread priority]. statProcessSwitch := statProcessSwitch + 1. activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer. objectMemory storePointer: SuspendedContextIndex ofObject: activeProc withValue: activeContext. self ensurePushedInstructionPointer. self externalWriteBackHeadFramePointers. self putToSleep: activeProc yieldingIf: preemptionYields. "Transcript cr; print: #cedeToHighestPriorityThreadIfHigherThan:; cr. self printExternalHeadFrame. self print: 'ip: '; printHex: self instructionPointer. Transcript cr; flush." self returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source: CSCheckEvents! Item was changed: ----- Method: CoInterpreterMT>>disownVM: (in category 'vm scheduling') ----- disownVM: flags "Release the VM to other threads and answer the current thread's index. Currently valid flags: DisownVMForFFICall - informs the VM that it is entering an FFI call DisownVMForThreading - informs the VM that it is entering code during which threading should be permitted OwnVMForeignThreadFlag - indicates lowest-level entry from a foreign thread - not to be used explicitly by clients - only set by ownVMFromUnidentifiedThread VMAlreadyOwnedHenceDoNotDisown - indicates an ownVM from a callback was made when the vm was still owned. - not to be used explicitly by clients - only set by ownVMFromUnidentifiedThread This is the entry-point for plugins and primitives that wish to release the VM while performing some operation that may potentially block, and for callbacks returning back to some blocking operation. If this thread does not reclaim the VM before- hand then when the next heartbeat occurs the thread manager will schedule a thread to acquire the VM which may start running the VM in place of this thread. N.B. Most of the state needed to resume after preemption is set in preemptDisowningThread." <public> <inline: false> <returnTypeC: #'void *'> + | vmThread activeProc | + self assert: flags >= 0. - | vmThread | - self assert: (flags >= 0 and: [flags < (1 bitShift: DisownFlagsShift)]). self assert: self successful. cogit recordEventTrace ifTrue: [self recordTrace: TraceDisownVM thing: (objectMemory integerObjectOf: flags) source: 0]. processHasThreadAffinity ifFalse: [willNotThreadWarnCount < 10 ifTrue: [self print: 'warning: VM parameter 48 indicates Process doesn''t have threadId; VM will not thread'; cr. willNotThreadWarnCount := willNotThreadWarnCount + 1]]. vmThread := cogThreadManager currentVMThread. (flags anyMask: VMAlreadyOwnedHenceDoNotDisown) ifTrue: [disowningVMThread := vmThread. vmThread setVmThreadState: CTMUnavailable. ^nil]. self assertCStackPointersBelongToCurrentThread. self assertValidNewMethodPropertyFlags. self cCode: '' inSmalltalk: [cogThreadManager saveRegisterStateForCurrentProcess. cogThreadManager clearRegisterStates.]. (flags anyMask: DisownVMForProcessorRelinquish) ifTrue: [| proc | (proc := objectMemory splObj: foreignCallbackProcessSlot) ~= objectMemory nilObject ifTrue: [foreignCallbackPriority := self quickFetchInteger: PriorityIndex ofObject: proc]. relinquishing := true. self sqLowLevelMFence]. disownCount := disownCount + 1. "If we're disowning the VM because there's no active process to run, there's nothing to preempt later, so don't indicate that there's a disowningVMThread that needs to be restored later." + activeProc := self activeProcess. + activeProc ~= objectMemory nilObject + ifTrue: [disowningVMThread := vmThread. + vmThread priority: (self quickFetchInteger: PriorityIndex ofObject: activeProc).]. - self activeProcess ~= objectMemory nilObject - ifTrue: [disowningVMThread := vmThread]. "OwnVMForeignThreadFlag indicates lowest-level of entry by a foreign thread. If that's where we are then release the vmThread. Otherwise indicate the vmThread is off doing something outside of the VM." (flags anyMask: OwnVMForeignThreadFlag) ifTrue: ["I don't think this is quite right. Josh's use case is creating some foreign thread and then registering it with the VM. That's not the same as binding a process to a foreign thread given that the foreign callback process is about to terminate anyway (it is returning from a callback here). So do we need an additional concept, that of a vmThread being either of the set known to the VM or floating?" self flag: 'issue with registering foreign threads with the VM'. (self isBoundProcess: self activeProcess) ifFalse: [cogThreadManager unregisterVMThread: vmThread]] ifFalse: [vmThread setVmThreadState: CTMUnavailable]. vmThread disownFlags: (flags bitOr: (activeProcessAffined ifTrue: [0] ifFalse: [ProcessUnaffinedOnDisown])). cogThreadManager releaseVM. ^vmThread! Item was added: + ----- Method: CoInterpreterMT>>getMaxWaitingPriority (in category 'accessing') ----- + getMaxWaitingPriority + + ^ self atomic_load: (self addressOf: maxWaitingPriority)! Item was changed: ----- Method: CoInterpreterMT>>initialize (in category 'initialization') ----- initialize super initialize. relinquishing := checkThreadActivation := deferThreadSwitch := false. + foreignCallbackPriority := disownCount := willNotThreadWarnCount := 0. + maxWaitingPriority := AtomicValue new.! - foreignCallbackPriority := maxWaitingPriority := disownCount := willNotThreadWarnCount := 0! Item was changed: ----- Method: CoInterpreterMT>>initializeInterpreter: (in category 'initialization') ----- initializeInterpreter: bytesToShift super initializeInterpreter: bytesToShift. foreignCallbackProcessSlot := (objectMemory lengthOf: objectMemory specialObjectsOop) > ForeignCallbackProcess ifTrue: [ForeignCallbackProcess] ifFalse: [NilObject]. + + self atomic_store: (self addressOf: maxWaitingPriority) _: 0.! - ! Item was changed: ----- Method: CoInterpreterMT>>ownVMFromUnidentifiedThread (in category 'vm scheduling') ----- ownVMFromUnidentifiedThread "Attempt to take ownership from a thread that as yet doesn't know its index. This supports callbacks where the callback could originate from any thread. Answer 0 if the owning thread is known to the VM. Answer 1 if the owning thread is unknown to the VM and now owns the VM. Answer -1 if the owning thread is unknown to the VM and fails to own the VM. Answer -2 if the owning thread is unknown to the VM and there is no foreign callback process installed." | count threadIndex vmThread | <var: #vmThread type: #'CogVMThread *'> <inline: false> self cCode: [] inSmalltalk: [self halt: 'TODO: Implement processor register switching']. (threadIndex := cogThreadManager ioGetThreadLocalThreadIndex) ~= 0 ifTrue: [ "this is a callback from a known thread" (cogThreadManager vmOwnerIs: threadIndex) ifTrue: "the VM has not been disowned" [self assert: (disowningVMThread isNil or: [disowningVMThread = self currentVMThread]). disowningVMThread := nil. self currentVMThread setVmThreadState: CTMAssignableOrInVM. + ^VMAlreadyOwnedHenceDoNotDisown]]. + + (threadIndex = 0 and: [foreignCallbackPriority = 0]) ifTrue: - ^VMAlreadyOwnedHenceDoNotDisown]. - ^self cCode: [nil] inSmalltalk: [self error: 'TODO: Needs to take a vmThread, not the index'."self ownVM: threadIndex"]]. - foreignCallbackPriority = 0 ifTrue: [^-2]. count := 0. + + "Before we can proceed, we need to temporarily lock the vm, so we can either find our CogVMThread struct or allocate a new one." + cogThreadManager acquireVMForIndex: threadIndex withPriority: foreignCallbackPriority. + + threadIndex ~= 0 + ifTrue: ["this is a callback from a known thread. Simply own the VM for that thread." + vmThread := cogThreadManager vmThreadAt: threadIndex. + ^ self ownVM: vmThread]. + "If the current thread doesn't have an index it's new to the vm and we need to allocate a new threadInfo, failing if we can't. We also need a process in the foreignCallbackProcessSlot upon + which to run the thread's eventual callback." + [(objectMemory splObj: foreignCallbackProcessSlot) = objectMemory nilObject] whileTrue: - which to run the thread's eventual callback." - [[cogThreadManager tryLockVMOwnerTo: cogThreadManager ioCurrentOSThread asUnsignedInteger] whileFalse: - [self waitingPriorityIsAtLeast: foreignCallbackPriority. - cogThreadManager ioTransferTimeslice]. - (objectMemory splObj: foreignCallbackProcessSlot) ~= objectMemory nilObject] whileFalse: [cogThreadManager releaseVM. (count := count + 1) > 1000 ifTrue: [^-2]. + cogThreadManager ioMilliSleep: 1. + cogThreadManager acquireVMForIndex: threadIndex withPriority: foreignCallbackPriority]. - cogThreadManager ioMilliSleep: 1]. vmThread := cogThreadManager unusedThreadInfo. "N.B. Keep the VM locked anonymously so that we reserve the non-nil ForeignCallbackProcess for this thread, avoiding the race between competing foreign callbacks. The acquireVMFor: in ownVM: will set the vmOwner to the actual index. So only unlock on failure." vmThread ifNil: [cogThreadManager releaseVM. ^-1]. cogThreadManager setVMOwner: vmThread index. vmThread + priority: foreignCallbackPriority; + disownFlags: OwnVMForeignThreadFlag; + setVmThreadState: CTMWantingOwnership. - setVmThreadState: CTMWantingOwnership; - priority: foreignCallbackPriority. cogThreadManager registerVMThread: vmThread. + ^self ownVM: vmThread! - ^self ownVM: vmThread index + OwnVMForeignThreadFlag! Item was changed: ----- Method: CoInterpreterMT>>preemptDisowningThread (in category 'vm scheduling') ----- preemptDisowningThread "Set the relevant state for disowningVMThread so that it can resume after being preempted and set disowningVMThread to nil to indicate preemption. N.B. This should only be sent from checkPreemptionOfDisowningThread. There are essentially four things to do. a) save the VM's notion of the current C stack pointers; these are pointers into a thread's stack and must be saved and restored in thread switch. b) save the VM's notion of the current Smalltalk execution point. This is simply the suspend half of a process switch that saves the current context in the current process. c) add the process to the thread's set of AWOL processes so that the scheduler won't try to run the process while the thread has disowned the VM. d) save the in-primitive VM state, newMethod and argumentCount ownVM: will restore the VM context as of disownVM: from the above when it finds it has been preempted." | activeProc activeContext preemptedThread | <var: #preemptedThread type: #'CogVMThread *'> <inline: false> self assert: disowningVMThread notNil. self assert: (disowningVMThread vmThreadState = CTMUnavailable or: [disowningVMThread vmThreadState = CTMWantingOwnership]). self assertCStackPointersBelongToDisowningThread. cogit recordEventTrace ifTrue: [self recordTrace: TracePreemptDisowningThread thing: (objectMemory integerObjectOf: disowningVMThread index) source: 0]. disowningVMThread cStackPointer: CStackPointer. disowningVMThread cFramePointer: CFramePointer. activeProc := self activeProcess. self assert: (objectMemory fetchPointer: MyListIndex ofObject: activeProc) = objectMemory nilObject. objectMemory storePointer: MyListIndex ofObject: activeProc withValue: (objectMemory splObj: ProcessInExternalCodeTag). activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer. objectMemory storePointer: SuspendedContextIndex ofObject: activeProc withValue: activeContext. "The instructionPointer must be pushed because the convention for inactive stack pages is that the instructionPointer is top of stack. We need to know if this primitive is called from machine code because the invariant that the return pc of an interpreter callee calling a machine code caller is ceReturnToInterpreterPC must be maintained." self push: instructionPointer. self externalWriteBackHeadFramePointers. "Since pushing the awol process may realloc disowningVMThread we need to reassign. But since we're going to nil disowningVMThread anyway we can assign to a local." preemptedThread := cogThreadManager pushAWOLProcess: activeProc on: disowningVMThread. disowningVMThread := nil. - preemptedThread priority: (self quickFetchInteger: PriorityIndex ofObject: activeProc). (self ownerIndexOfProcess: activeProc) = 0 ifTrue: [self setOwnerIndexOfProcess: activeProc to: preemptedThread index bind: false]. preemptedThread newMethodOrNull: newMethod; argumentCount: argumentCount; inMachineCode: instructionPointer <= objectMemory startOfMemory! Item was changed: ----- Method: CoInterpreterMT>>primitiveProcessBindToThreadId (in category 'process primitives') ----- primitiveProcessBindToThreadId "Attempt to bind the receiver to the thread with the id of the argument or nil, where the receiver is a Process. If successful the VM will ensure that there are at least id many threads active." + | aProcess id waitingPriority activePriority | - | aProcess id | <export: true> self cCode: [] inSmalltalk: [cogThreadManager isNil ifTrue: [^self primitiveFail]]. processHasThreadAffinity ifFalse: [^self primitiveFailFor: PrimErrUnsupported]. id := self stackTop. aProcess := self stackValue: 1. ((id = objectMemory nilObject or: [(objectMemory isIntegerObject: id) and: [id ~= (objectMemory integerObjectOf: 0)]]) and: [(objectMemory isPointers: aProcess) and: [(objectMemory slotSizeOf: aProcess) >= (ThreadIdIndex + 1)]]) ifFalse: [^self primitiveFailFor: PrimErrBadArgument]. id := id = objectMemory nilObject ifTrue: [0] ifFalse: [objectMemory integerValueOf: id]. id abs >= cogThreadManager maxNumThreads ifTrue: [^self primitiveFailFor: PrimErrLimitExceeded]. (self bindProcess: aProcess toId: id) ifNotNil: [:ec| ^self primitiveFailFor: ec]. self methodReturnReceiver. + waitingPriority := self getMaxWaitingPriority. + activePriority := self quickFetchInteger: PriorityIndex ofObject: aProcess. id := self ownerIndexOfProcess: aProcess. (aProcess = self activeProcess and: [(activeProcessAffined := id ~= 0) and: [(cogThreadManager vmOwnerIsCompatibleWith: id) not]]) ifTrue: + [activePriority < waitingPriority ifTrue: + [self reduceWaitingPriorityFrom: waitingPriority to: activePriority "TODO: Check if this is correct?"]. - [(self quickFetchInteger: PriorityIndex ofObject: aProcess) < maxWaitingPriority ifTrue: - [maxWaitingPriority = self quickFetchInteger: PriorityIndex ofObject: aProcess]. self threadSwitchIfNecessary: aProcess from: CSThreadBind]! Item was added: + ----- Method: CoInterpreterMT>>reduceWaitingPriorityFrom:to: (in category 'accessing') ----- + reduceWaitingPriorityFrom: existingWaitingPriority to: newMaxPriority + <var: #existing type: #int> + | existing | + self cCode: [existing := existingWaitingPriority] + inSmalltalk: [existing := AtomicValue new. + existing value: existingWaitingPriority.]. + "This CPXCHG may fail, that's fine though, as there may have been + another thread that increased the priority in the meantime. + In that case that threads priority is the correct one to use." + ^ self atomic: (self addressOf: maxWaitingPriority) + _compare: (self addressOf: existing) + _exchange_strong: newMaxPriority! Item was removed: - ----- Method: CoInterpreterMT>>setMaxWaitingPriorityTo: (in category 'process primitive support') ----- - setMaxWaitingPriorityTo: minPriority - maxWaitingPriority := minPriority! Item was changed: ----- Method: CoInterpreterMT>>threadSchedulingLoopImplementation: (in category 'vm scheduling') ----- threadSchedulingLoopImplementation: vmThread "Enter a loop attempting to run the VM with the highest priority process and blocking on the thread's OS semaphore when unable to run that process. We will return to this via threadSwitchIfNecessary:from: which is called in the middle of transferTo:from: once the active process has been stored in the scheduler." <var: #vmThread type: #'CogVMThread *'> <inline: false> self _setjmp: vmThread reenterThreadSchedulingLoop. [self assert: vmThread vmThreadState = CTMAssignableOrInVM. (cogThreadManager tryLockVMOwnerTo: vmThread index) ifTrue: ["Yay, we're the VM owner!!" "If relinquishing is true, then primitiveRelinquishProcessor has disowned the VM and only a returning call or callback should take ownership in that case." relinquishing ifFalse: [self tryToExecuteSmalltalk: vmThread]. + "tryToExecuteSmalltalk: may return if there's no runnable process. + Usually it doesn't return, but jumps straight back to the _setjmp at the top of this function, + so this is only reached in case there's no runnable process." + self disownVM: DisownVMForThreading]. - self disownVM: DisownVMForThreading.]. cogThreadManager waitForWork: vmThread. true] whileTrue! Item was changed: ----- Method: CoInterpreterMT>>threadSwitchIfNecessary:from: (in category 'process primitive support') ----- threadSwitchIfNecessary: newProc from: sourceCode "Invoked from transferTo:from: or primitiveProcessBindToThreadId to switch threads if the new process is bound or affined to some other thread." | newProcOwnerIndex vmThread activeContext | self assert: (cogThreadManager vmOwnerIs: cogThreadManager ioGetThreadLocalThreadIndex). deferThreadSwitch ifTrue: [^self]. cogThreadManager assertValidProcessorStackPointersForIndex: cogThreadManager getVMOwner. "If the current process is unaffined or it is affined to the current thread we're ok to run, but we should yield asap if a higher-priority thread wants the VM." newProcOwnerIndex := self ownerIndexOfProcess: newProc. ((activeProcessAffined := newProcOwnerIndex ~= 0) and: [(cogThreadManager vmOwnerIsCompatibleWith: newProcOwnerIndex) not]) ifFalse: + [(self quickFetchInteger: PriorityIndex ofObject: newProc) < self getMaxWaitingPriority ifTrue: - [(self quickFetchInteger: PriorityIndex ofObject: newProc) < maxWaitingPriority ifTrue: [checkThreadActivation := true. self forceInterruptCheck]. ^self]. "The current process is affined to a thread, but not to the current owner. So switch to that owner." self cCode: '' inSmalltalk: [transcript ensureCr; f: 'threadSwitchIfNecessary: %08x from: %s(%d) owner %d -> %d\n' printf: { newProc. TraceSources at: sourceCode. sourceCode. cogThreadManager getVMOwner. newProcOwnerIndex }]. "We at least need to externalize the stack pointers to enable a thread switch..." (objectMemory fetchPointer: SuspendedContextIndex ofObject: newProc) = objectMemory nilObject ifTrue: [self assert: newProc = self activeProcess. self push: instructionPointer. self externalWriteBackHeadFramePointers. false ifTrue: "If the activeProcess doesn't have a context yet, it needs one from which the new thread can resume execution." [activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer. objectMemory storePointer: SuspendedContextIndex ofObject: newProc withValue: activeContext]]. newProcOwnerIndex < 0 ifTrue: [self assert: newProcOwnerIndex negated = cogThreadManager getVMOwner. vmThread := cogThreadManager ensureWillingThread. self deny: vmThread index = cogThreadManager getVMOwner. self assert: (cogThreadManager threadIndex: vmThread index isCompatibleWith: newProcOwnerIndex)] ifFalse: [vmThread := cogThreadManager vmThreadAt: newProcOwnerIndex. vmThread priority: (self quickFetchInteger: PriorityIndex ofObject: newProc). vmThread vmThreadState = CTMUnavailable ifTrue: [vmThread setVmThreadState: CTMWantingOwnership]]. self returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source: CSSwitchIfNeccessary! Item was changed: + ----- Method: CoInterpreterMT>>waitingPriorityIsAtLeast: (in category 'accessing') ----- - ----- Method: CoInterpreterMT>>waitingPriorityIsAtLeast: (in category 'process primitive support') ----- waitingPriorityIsAtLeast: minPriority "Set the maxWaitingPriority to at least minPriority on behalf of a thread wanting to acquire the VM. If maxWaitingPriority is increased, schedule a thread activation check asap." + <var: #currentWaitingPriority type: #int> + | currentWaitingPriority didIncrease | + self cCode: [currentWaitingPriority := self getMaxWaitingPriority.] + inSmalltalk: [currentWaitingPriority := AtomicValue new. + currentWaitingPriority value: self getMaxWaitingPriority]. + + didIncrease := false. + [(self cCode: [currentWaitingPriority] inSmalltalk: [currentWaitingPriority value]) >= minPriority + or: [didIncrease := self atomic: (self addressOf: maxWaitingPriority) + _compare: (self addressOf: currentWaitingPriority) + _exchange_strong: minPriority]] whileFalse. + didIncrease + ifTrue: [ self assert: currentWaitingPriority < minPriority. + checkThreadActivation := true. + self forceInterruptCheck]! - maxWaitingPriority < minPriority ifTrue: - [maxWaitingPriority := minPriority. - checkThreadActivation := true. - self forceInterruptCheck]! Item was changed: ----- Method: CogThreadManager class>>declareCVarsIn: (in category 'translation') ----- declareCVarsIn: cCodeGen cCodeGen removeVariable: 'coInterpreter'; removeVariable: 'cogit'; removeVariable: 'threadLocalStorage'; removeVariable: 'processorOwner'; removeVariable: 'registerStates'. cCodeGen var: #threads type: #'CogVMThread **'; var: #vmOSThread type: #sqOSThread; + var: #vmOwner type: #'volatile atomic_int'; + var: #maxWaitingPriority type: #'volatile atomic_int'. - var: #vmOwner type: #'volatile atomic_int'. cCodeGen addHeaderFile: '<stdatomic.h>'! Item was changed: ----- Method: CogThreadManager>>acquireVMFor: (in category 'public api') ----- acquireVMFor: vmThread "Attempt to acquire the VM, eventually blocking until it becomes available. Spin until the maxWaitingPriority has been updated if it is lower than this thread's priority." <returnTypeC: #'CogVMThread *'> <var: #vmThread type: #'CogVMThread *'> self assert: vmThread index = self ioGetThreadLocalThreadIndex. self assert: (vmThread vmThreadState = CTMUnavailable or: [vmThread vmThreadState = CTMWantingOwnership]). (self tryLockVMOwnerTo: vmThread index) ifTrue: [vmThread setVmThreadState: CTMAssignableOrInVM] ifFalse: [vmThread setVmThreadState: CTMWantingOwnership. + [self tryLockVMOwnerTo: vmThread index] whileFalse: - [(self vmOwnerIs: vmThread index) - or: [self tryLockVMOwnerTo: vmThread index]] whileFalse: [vmThread priority ifNotNil: [coInterpreter waitingPriorityIsAtLeast: vmThread priority]. (self vmOwnerIs: vmThread index) ifFalse: [self ioWaitOnOSSemaphore: (self addressOf: vmThread osSemaphore)]]]. coInterpreter assertProcessorStackPointersBelongToCurrentThread. vmOSThread := vmThread osThread. vmThread setVmThreadState: CTMAssignableOrInVM. ^vmThread! Item was added: + ----- Method: CogThreadManager>>acquireVMForIndex:withPriority: (in category 'public api') ----- + acquireVMForIndex: threadIndex withPriority: aPriority + + [self tryLockVMOwnerTo: (threadIndex > 0 ifTrue: [threadIndex] ifFalse: [CTMUnknownOwner])] whileFalse: + [self waitingPriorityIsAtLeast: aPriority. + self ioTransferTimeslice].! Item was changed: + ----- Method: CogThreadManager>>doTryLockVMOwnerTo: (in category 'Cogit lock implementation') ----- - ----- Method: CogThreadManager>>doTryLockVMOwnerTo: (in category 'simulation') ----- doTryLockVMOwnerTo: threadIndex "In the simulation, this is being called by #simulateTryLockVMOwnerTo:, in C this method will just be called directly. Returns true if the vmOwner has been successfully set to the given thread index." <inline: #always> <var: #expected type: #int> | expected | expected := self cCode: 0 inSmalltalk: [AtomicValue newFrom: 0]. ^ (self atomic: (self addressOf: vmOwner) _compare: (self addressOf: expected) _exchange_strong: threadIndex) or: ["We may already be vmOwner. The current vmOwner will be stored in expected. However, if an unknown owner is present, we cannot assume that's us!!" expected = threadIndex and: [threadIndex ~= CTMUnknownOwner]]! Item was changed: ----- Method: CogThreadManager>>highestPriorityThreadIfHigherThan:expectedMax: (in category 'public api') ----- highestPriorityThreadIfHigherThan: activePriority expectedMax: maxPriority "Answer the first vmThread waiting to acquire the VM that is of higher priority than activePriority, or answer nil if none. If there is a higher priority thread then set the coInterpreter's maxWaitingPriority to either the priority of the next highest priority vmThread, or to 0 if none is waiting." <returnTypeC: #'CogVMThread *'> | vmThread highest nextHighest | <var: #vmThread type: #'CogVMThread *'> <var: #highest type: #'CogVMThread *'> <var: #nextHighest type: #'CogVMThread *'> highest := nextHighest := nil. "To make this fair we could remember the last index at which we found the highest and start the search at the following index." 1 to: numThreads do: [:i| vmThread := threads at: i. vmThread vmThreadState = CTMWantingOwnership ifTrue: [self assert: vmThread priority <= maxPriority. highest isNil ifTrue: [highest := vmThread] ifFalse: [vmThread priority > highest priority ifTrue: [nextHighest := highest. highest := vmThread] ifFalse: [nextHighest isNil ifTrue: [nextHighest := vmThread] ifFalse: [vmThread priority > nextHighest priority ifTrue: [nextHighest := vmThread]]]]]]. highest isNil ifTrue: [^nil]. highest priority <= activePriority ifTrue: [^nil]. + coInterpreter waitingPriorityIsAtLeast: (nextHighest isNil - coInterpreter setMaxWaitingPriorityTo: (nextHighest isNil ifTrue: [0] ifFalse: [nextHighest priority]). ^highest! Item was removed: - ----- Method: CogVMSimulator>>setMaxWaitingPriorityTo: (in category 'multi-threading simulation switch') ----- - setMaxWaitingPriorityTo: minPriority - "This method includes or excludes CoInterpreterMT methods as required. - Auto-generated by CogVMSimulator>>ensureMultiThreadingOverridesAreUpToDate" - - ^self perform: #setMaxWaitingPriorityTo: - withArguments: {minPriority} - inSuperclass: (cogThreadManager ifNil: [CoInterpreterPrimitives] ifNotNil: [CoInterpreterMT])!

1 0

VM Maker: VMMaker.threaded-LM.3342.mcz
by commits＠source.squeak.org 10 Oct '23

10 Oct '23

1 0

VM Maker: VMMaker.threaded-LM.3341.mcz
by commits＠source.squeak.org 05 Oct '23

05 Oct '23

Leon Matthes uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.threaded-LM.3341.mcz ==================== Summary ==================== Name: VMMaker.threaded-LM.3341 Author: LM Time: 3 October 2023, 7:03:09.087577 pm UUID: ec2245e8-492b-4c66-9f42-a836ba795a7d Ancestors: VMMaker.threaded-LM.3339 Allow -1 to be set as vmOwner. This is currently used by the heartbeat thread. In future this will be used by the ownVMFromUnidentifiedThread function. This combines well with the additional change that makes disownVM return a CogVMThread* disguised as void*. We can then ensure that the threads variable is only ever accessed by the vm owner. =============== Diff against VMMaker.threaded-LM.3339 =============== Item was added: + ----- Method: AtomicValue>>printOn: (in category 'as yet unclassified') ----- + printOn: aStream + + aStream nextPutAll: 'Atomic: '. + self value printOn: aStream.! Item was changed: ----- Method: CCodeGenerator>>addStructClass: (in category 'public') ----- addStructClass: aClass "Add the non-accessor methods of the given struct class to the code base." aClass prepareToBeAddedToCodeGenerator: self. self addClassVarsFor: aClass. self addPoolVarsFor: aClass. self retainMethods: (aClass requiredMethodNames: self options). 'Adding Class ' , aClass name , '...' displayProgressAt: Sensor cursorPoint from: 0 to: aClass selectors size during: [:bar | aClass selectors withIndexDo: [:sel :i | | source | bar value: i. self addStructMethodFor: aClass selector: sel]]. aClass declareCVarsIn: self! Item was changed: ----- Method: CCodeGenerator>>addStructMethodFor:selector: (in category 'utilities') ----- addStructMethodFor: aClass selector: selector "Add the given struct method to the code base and answer its translation or nil if it shouldn't be translated." (self methodNamed: selector) ifNotNil: [:tmethod| "If we're repeating an attempt to add the same thing, or if the existing method overrides this one,don't complain." (tmethod definingClass includesBehavior: aClass) ifTrue: [^self]. "If the methods are both simple accessors, don't complain." ((tmethod definingClass isAccessor: selector) and: [aClass isAccessor: selector]) ifTrue: [^self]. "If the method is overriding a method in a superclass, don't complain" (aClass inheritsFrom: tmethod definingClass) ifTrue: [methods removeKey: selector] ifFalse: [self error: 'conflicting implementations for ', selector storeString]]. ^(self addMethodFor: aClass selector: selector) ifNotNil: [:tmethod| tmethod transformToStructClassMethodFor: self. tmethod]! Item was changed: ----- Method: CCodeGenerator>>doInlining: (in category 'inlining') ----- doInlining: inlineFlagOrSymbol "Inline the bodies of all methods that are suitable for inlining." "Modified slightly for the core VM translator, since the first level of inlining for the interpret loop must be performed in order that the instruction implementations can easily discover their addresses. Remember to inline the bytecode routines as well" | removed | beganInlining := true. inlineFlagOrSymbol isSymbol ifTrue: [self inlineDispatchesInMethodNamed: #interpret localizingVars: #(). self doBasicInlining: inlineFlagOrSymbol. self pruneUnreachableMethods. ^self]. inlineFlagOrSymbol ifFalse: [self inlineDispatchesInMethodNamed: #interpret localizingVars: #(). self pruneUnreachableMethods. ^self]. self doBasicInlining: inlineFlagOrSymbol. vmClass ifNil: [^self]. 'Inlining bytecodes' displayProgressAt: Sensor cursorPoint from: 1 to: 2 during: [:bar | self inlineDispatchesInMethodNamed: #interpret localizingVars: vmClass namesOfVariablesToLocalize. bar value: 1. removed := self removeMethodsReferingToGlobals: vmClass namesOfVariablesToLocalize except: #interpret. bar value: 2]. "only prune when generating the interpreter itself" self pruneUnreachableMethods. self reportShouldNotBeRemoved: removed varList: vmClass namesOfVariablesToLocalize! Item was changed: ----- Method: CoInterpreterMT>>checkVMOwnershipFromHeartbeat (in category 'process primitive support') ----- checkVMOwnershipFromHeartbeat "Check whether the VM is unowned and needs to set a thread running to try and own it. Do not attempt this if the image doesn't have a threadAffinity inst var in Process; the VM can't thread these images." <inline: false> self sqLowLevelMFence. (processHasThreadAffinity + and: [cogThreadManager doTryLockVMOwnerTo: CTMUnknownOwner]) ifTrue: - and: [cogThreadManager vmIsUnowned]) ifTrue: [cogThreadManager ensureRunningVMThread: relinquishing]! Item was changed: ----- Method: CoInterpreterMT>>disownVM: (in category 'vm scheduling') ----- disownVM: flags "Release the VM to other threads and answer the current thread's index. Currently valid flags: DisownVMForFFICall - informs the VM that it is entering an FFI call DisownVMForThreading - informs the VM that it is entering code during which threading should be permitted OwnVMForeignThreadFlag - indicates lowest-level entry from a foreign thread - not to be used explicitly by clients - only set by ownVMFromUnidentifiedThread VMAlreadyOwnedHenceDoNotDisown - indicates an ownVM from a callback was made when the vm was still owned. - not to be used explicitly by clients - only set by ownVMFromUnidentifiedThread This is the entry-point for plugins and primitives that wish to release the VM while performing some operation that may potentially block, and for callbacks returning back to some blocking operation. If this thread does not reclaim the VM before- hand then when the next heartbeat occurs the thread manager will schedule a thread to acquire the VM which may start running the VM in place of this thread. N.B. Most of the state needed to resume after preemption is set in preemptDisowningThread." <public> <inline: false> + <returnTypeC: #'void *'> + | vmThread | - | vmThread result | self assert: (flags >= 0 and: [flags < (1 bitShift: DisownFlagsShift)]). self assert: self successful. cogit recordEventTrace ifTrue: [self recordTrace: TraceDisownVM thing: (objectMemory integerObjectOf: flags) source: 0]. processHasThreadAffinity ifFalse: [willNotThreadWarnCount < 10 ifTrue: [self print: 'warning: VM parameter 48 indicates Process doesn''t have threadId; VM will not thread'; cr. willNotThreadWarnCount := willNotThreadWarnCount + 1]]. vmThread := cogThreadManager currentVMThread. (flags anyMask: VMAlreadyOwnedHenceDoNotDisown) ifTrue: [disowningVMThread := vmThread. vmThread setVmThreadState: CTMUnavailable. + ^nil]. - ^0]. self assertCStackPointersBelongToCurrentThread. self assertValidNewMethodPropertyFlags. self cCode: '' inSmalltalk: [cogThreadManager saveRegisterStateForCurrentProcess. cogThreadManager clearRegisterStates.]. (flags anyMask: DisownVMForProcessorRelinquish) ifTrue: [| proc | (proc := objectMemory splObj: foreignCallbackProcessSlot) ~= objectMemory nilObject ifTrue: [foreignCallbackPriority := self quickFetchInteger: PriorityIndex ofObject: proc]. relinquishing := true. self sqLowLevelMFence]. disownCount := disownCount + 1. "If we're disowning the VM because there's no active process to run, there's nothing to preempt later, so don't indicate that there's a disowningVMThread that needs to be restored later." self activeProcess ~= objectMemory nilObject ifTrue: [disowningVMThread := vmThread]. "OwnVMForeignThreadFlag indicates lowest-level of entry by a foreign thread. If that's where we are then release the vmThread. Otherwise indicate the vmThread is off doing something outside of the VM." (flags anyMask: OwnVMForeignThreadFlag) ifTrue: ["I don't think this is quite right. Josh's use case is creating some foreign thread and then registering it with the VM. That's not the same as binding a process to a foreign thread given that the foreign callback process is about to terminate anyway (it is returning from a callback here). So do we need an additional concept, that of a vmThread being either of the set known to the VM or floating?" self flag: 'issue with registering foreign threads with the VM'. (self isBoundProcess: self activeProcess) ifFalse: [cogThreadManager unregisterVMThread: vmThread]] ifFalse: [vmThread setVmThreadState: CTMUnavailable]. + vmThread disownFlags: (flags bitOr: (activeProcessAffined ifTrue: [0] ifFalse: [ProcessUnaffinedOnDisown])). - result := ((vmThread index bitShift: DisownFlagsShift) - bitOr: (activeProcessAffined ifTrue: [0] ifFalse: [ProcessUnaffinedOnDisown])) - bitOr: flags. cogThreadManager releaseVM. + ^vmThread! - ^result! Item was changed: ----- Method: CoInterpreterMT>>ownVM: (in category 'vm scheduling') ----- + ownVM: vmThreadHandle - ownVM: threadIndexAndFlags <public> <inline: false> + <var: #vmThreadHandle type: #'void *'> + <var: #vmThread type: #'CogVMThread *'> "This is the entry-point for plugins and primitives that wish to reacquire the VM after having released it via disownVM or callbacks that want to acquire it without knowing their ownership status. This call will block until the VM is owned by the current thread or an error occurs. The argument should be the value answered by disownVM, or 0 for callbacks that don't know if they have disowned or not. This is both an optimization to avoid having to query thread- local storage for the current thread's index (since it can easily keep it in some local variable), and a record of when an unbound process becomes affined to a thread for the dynamic extent of some operation. Answer 0 if the current thread is known to the VM (and on return owns the VM). Answer 1 if the current thread is unknown to the VM and takes ownership. Answer -1 if the current thread is unknown to the VM and fails to take ownership." + | flags vmThread | + vmThread := self cCoerce: vmThreadHandle to: #'CogVMThread *'. + vmThread ifNil: - | threadIndex flags vmThread | - threadIndexAndFlags = 0 ifTrue: [^self ownVMFromUnidentifiedThread]. + flags := vmThread disownFlags. - threadIndex := threadIndexAndFlags bitShift: DisownFlagsShift negated. - flags := threadIndexAndFlags bitAnd: (1 bitShift: DisownFlagsShift) - 1. (flags anyMask: DisownVMForProcessorRelinquish) ifTrue: ["Presumably we have nothing to do; this primitive is typically called from the background process. So we should /not/ try and activate any threads in the pool; they will waste cycles finding there is no runnable process, and will cause a VM abort if no runnable process is found. But we /do/ want to allow FFI calls that have completed, or callbacks a chance to get into the VM; they do have something to do. DisownVMForProcessorRelinquish indicates this." relinquishing := false. self sqLowLevelMFence]. + vmThread := cogThreadManager acquireVMFor: vmThread. - vmThread := cogThreadManager acquireVMFor: threadIndex. disownCount := disownCount - 1. disowningVMThread ifNotNil: [vmThread = disowningVMThread ifTrue: [self assert: (vmThread cFramePointer isNil or: [CFramePointer = vmThread cFramePointer and: [CStackPointer = vmThread cStackPointer]]). self assert: self successful. self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. disowningVMThread := nil. cogit recordEventTrace ifTrue: [self recordTrace: TraceOwnVM thing: ConstOne source: 0]. ^0]. "if not preempted we're done." self preemptDisowningThread]. "We've been preempted; we must restore state and update the threadId in our process, and may have to put the active process to sleep." + self restoreVMStateFor: vmThread andFlags: flags. - self restoreVMStateFor: vmThread threadIndexAndFlags: threadIndexAndFlags. cogit recordEventTrace ifTrue: [self recordTrace: TraceOwnVM thing: ConstTwo source: 0]. + ^flags bitAnd: OwnVMForeignThreadFlag! - ^threadIndexAndFlags bitAnd: OwnVMForeignThreadFlag! Item was changed: ----- Method: CoInterpreterMT>>ownVMFromUnidentifiedThread (in category 'vm scheduling') ----- ownVMFromUnidentifiedThread "Attempt to take ownership from a thread that as yet doesn't know its index. This supports callbacks where the callback could originate from any thread. Answer 0 if the owning thread is known to the VM. Answer 1 if the owning thread is unknown to the VM and now owns the VM. Answer -1 if the owning thread is unknown to the VM and fails to own the VM. Answer -2 if the owning thread is unknown to the VM and there is no foreign callback process installed." | count threadIndex vmThread | <var: #vmThread type: #'CogVMThread *'> <inline: false> self cCode: [] inSmalltalk: [self halt: 'TODO: Implement processor register switching']. (threadIndex := cogThreadManager ioGetThreadLocalThreadIndex) ~= 0 ifTrue: [ "this is a callback from a known thread" (cogThreadManager vmOwnerIs: threadIndex) ifTrue: "the VM has not been disowned" [self assert: (disowningVMThread isNil or: [disowningVMThread = self currentVMThread]). disowningVMThread := nil. self currentVMThread setVmThreadState: CTMAssignableOrInVM. ^VMAlreadyOwnedHenceDoNotDisown]. + ^self cCode: [nil] inSmalltalk: [self error: 'TODO: Needs to take a vmThread, not the index'."self ownVM: threadIndex"]]. - ^self ownVM: threadIndex]. foreignCallbackPriority = 0 ifTrue: [^-2]. count := 0. "If the current thread doesn't have an index it's new to the vm and we need to allocate a new threadInfo, failing if we can't. We also need a process in the foreignCallbackProcessSlot upon which to run the thread's eventual callback." [[cogThreadManager tryLockVMOwnerTo: cogThreadManager ioCurrentOSThread asUnsignedInteger] whileFalse: [self waitingPriorityIsAtLeast: foreignCallbackPriority. cogThreadManager ioTransferTimeslice]. (objectMemory splObj: foreignCallbackProcessSlot) ~= objectMemory nilObject] whileFalse: [cogThreadManager releaseVM. (count := count + 1) > 1000 ifTrue: [^-2]. cogThreadManager ioMilliSleep: 1]. vmThread := cogThreadManager unusedThreadInfo. "N.B. Keep the VM locked anonymously so that we reserve the non-nil ForeignCallbackProcess for this thread, avoiding the race between competing foreign callbacks. The acquireVMFor: in ownVM: will set the vmOwner to the actual index. So only unlock on failure." vmThread ifNil: [cogThreadManager releaseVM. ^-1]. cogThreadManager setVMOwner: vmThread index. vmThread setVmThreadState: CTMWantingOwnership; priority: foreignCallbackPriority. cogThreadManager registerVMThread: vmThread. ^self ownVM: vmThread index + OwnVMForeignThreadFlag! Item was changed: ----- Method: CoInterpreterMT>>primitiveRelinquishProcessor (in category 'I/O primitives') ----- primitiveRelinquishProcessor "Relinquish the processor for up to the given number of microseconds. The exact behavior of this primitive is platform dependent. Override to check for waiting threads." + | microSecs vmHandle currentCStackPointer currentCFramePointer | - | microSecs threadIndexAndFlags currentCStackPointer currentCFramePointer | <var: #currentCStackPointer type: #'volatile usqIntptr_t'> <var: #currentCFramePointer type: #'volatile usqIntptr_t'> microSecs := self stackTop. (objectMemory isIntegerObject: microSecs) ifFalse: [^self primitiveFail]. self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. self assert: relinquishing not. "DO NOT allow relinquishing the processor while we are profiling since this may skew the time base for our measures (it may reduce processor speed etc). Instead we go full speed, therefore measuring the precise time we spend in the inner idle loop as a busy loop." nextProfileTick = 0 ifTrue: "Presumably we have nothing to do; this primitive is typically called from the background process. So we should /not/ try and activate any threads in the pool; they will waste cycles finding there is no runnable process, and will cause a VM abort if no runnable process is found. But we /do/ want to allow FFI calls that have completed, or callbacks a chance to get into the VM; they do have something to do. DisownVMForProcessorRelinquish indicates this." [currentCStackPointer := CStackPointer. currentCFramePointer := CFramePointer. + vmHandle := self disownVM: DisownVMForProcessorRelinquish. - threadIndexAndFlags := self disownVM: DisownVMForProcessorRelinquish. self assert: relinquishing. self ioRelinquishProcessorForMicroseconds: (objectMemory integerValueOf: microSecs). self assert: relinquishing. + self ownVM: vmHandle. - self ownVM: threadIndexAndFlags. self assert: relinquishing not. self assert: cogThreadManager currentVMThread vmThreadState = CTMAssignableOrInVM. self assert: currentCStackPointer = CStackPointer. self assert: currentCFramePointer = CFramePointer. "In simulation we allow ioRelinquishProcessorForMicroseconds: to fail so that we can arrange that the simulator responds to input events promptly. This *DOES NOT HAPPEN* in the real vm." self cCode: [] inSmalltalk: [primFailCode ~= 0 ifTrue: [^self]]]. self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. self pop: 1 "microSecs; leave rcvr on stack"! Item was added: + ----- Method: CoInterpreterMT>>restoreVMStateFor:andFlags: (in category 'vm scheduling') ----- + restoreVMStateFor: vmThread andFlags: flags + "We've been preempted; we must restore state and update the threadId + in our process, and may have to put the active process to sleep." + | sched activeProc myProc | + sched := self schedulerPointer. + activeProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched. + (flags anyMask: OwnVMForeignThreadFlag) + ifTrue: + [self assert: foreignCallbackProcessSlot == ForeignCallbackProcess. + myProc := objectMemory splObj: foreignCallbackProcessSlot. + self assert: myProc ~= objectMemory nilObject. + objectMemory splObj: foreignCallbackProcessSlot put: objectMemory nilObject] + ifFalse: [myProc := cogThreadManager popAWOLProcess: vmThread]. + self assert: activeProc ~= myProc. + (activeProc ~= objectMemory nilObject + and: [(objectMemory fetchPointer: MyListIndex ofObject: activeProc) = objectMemory nilObject]) ifTrue: + [self putToSleep: activeProc yieldingIf: preemptionYields]. + self assert: (objectMemory fetchPointer: MyListIndex ofObject: myProc) = (objectMemory splObj: ProcessInExternalCodeTag). + objectMemory + storePointer: ActiveProcessIndex ofObject: sched withValue: myProc; + storePointerUnchecked: MyListIndex ofObject: myProc withValue: objectMemory nilObject. + "Only unaffine if the process was affined at this level and did not become bound in the interim." + ((flags anyMask: ProcessUnaffinedOnDisown) + and: [(self isBoundProcess: myProc) not]) ifTrue: + [self setOwnerIndexOfProcess: myProc to: 0 bind: false]. + self initPrimCall. + self cCode: + [self externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc] + inSmalltalk: + ["Bypass the no-offset stack depth check in the simulator's externalSetStackPageAndPointersForSuspendedContextOfProcess:" + super externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc. + "We're in ownVM:, hence in a primitive, hence need to include the argument count" + (self isMachineCodeFrame: framePointer) ifTrue: + [self maybeCheckStackDepth: vmThread argumentCount + sp: stackPointer + pc: instructionPointer]]. + "If this primitive is called from machine code maintain the invariant that the return pc + of an interpreter callee calling a machine code caller is ceReturnToInterpreterPC." + (vmThread inMachineCode + and: [instructionPointer >= objectMemory startOfMemory]) ifTrue: + [self iframeSavedIP: framePointer put: instructionPointer. + instructionPointer := cogit ceReturnToInterpreterPC]. + newMethod := vmThread newMethodOrNull. + argumentCount := vmThread argumentCount. + vmThread newMethodOrNull: nil. + self cCode: '' inSmalltalk: + [| range | + range := self cStackRangeForThreadIndex: vmThread index. + self assert: ((range includes: vmThread cStackPointer) and: [range includes: vmThread cFramePointer])]. + self setCFramePointer: vmThread cFramePointer setCStackPointer: vmThread cStackPointer. + self assert: newMethod notNil + ! Item was removed: - ----- Method: CoInterpreterMT>>restoreVMStateFor:threadIndexAndFlags: (in category 'vm scheduling') ----- - restoreVMStateFor: vmThread threadIndexAndFlags: threadIndexAndFlags - "We've been preempted; we must restore state and update the threadId - in our process, and may have to put the active process to sleep." - | sched activeProc myProc | - sched := self schedulerPointer. - activeProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched. - (threadIndexAndFlags anyMask: OwnVMForeignThreadFlag) - ifTrue: - [self assert: foreignCallbackProcessSlot == ForeignCallbackProcess. - myProc := objectMemory splObj: foreignCallbackProcessSlot. - self assert: myProc ~= objectMemory nilObject. - objectMemory splObj: foreignCallbackProcessSlot put: objectMemory nilObject] - ifFalse: [myProc := cogThreadManager popAWOLProcess: vmThread]. - self assert: activeProc ~= myProc. - (activeProc ~= objectMemory nilObject - and: [(objectMemory fetchPointer: MyListIndex ofObject: activeProc) = objectMemory nilObject]) ifTrue: - [self putToSleep: activeProc yieldingIf: preemptionYields]. - self assert: (objectMemory fetchPointer: MyListIndex ofObject: myProc) = (objectMemory splObj: ProcessInExternalCodeTag). - objectMemory - storePointer: ActiveProcessIndex ofObject: sched withValue: myProc; - storePointerUnchecked: MyListIndex ofObject: myProc withValue: objectMemory nilObject. - "Only unaffine if the process was affined at this level and did not become bound in the interim." - ((threadIndexAndFlags anyMask: ProcessUnaffinedOnDisown) - and: [(self isBoundProcess: myProc) not]) ifTrue: - [self setOwnerIndexOfProcess: myProc to: 0 bind: false]. - self initPrimCall. - self cCode: - [self externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc] - inSmalltalk: - ["Bypass the no-offset stack depth check in the simulator's externalSetStackPageAndPointersForSuspendedContextOfProcess:" - super externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc. - "We're in ownVM:, hence in a primitive, hence need to include the argument count" - (self isMachineCodeFrame: framePointer) ifTrue: - [self maybeCheckStackDepth: vmThread argumentCount - sp: stackPointer - pc: instructionPointer]]. - "If this primitive is called from machine code maintain the invariant that the return pc - of an interpreter callee calling a machine code caller is ceReturnToInterpreterPC." - (vmThread inMachineCode - and: [instructionPointer >= objectMemory startOfMemory]) ifTrue: - [self iframeSavedIP: framePointer put: instructionPointer. - instructionPointer := cogit ceReturnToInterpreterPC]. - newMethod := vmThread newMethodOrNull. - argumentCount := vmThread argumentCount. - vmThread newMethodOrNull: nil. - self cCode: '' inSmalltalk: - [| range | - range := self cStackRangeForThreadIndex: vmThread index. - self assert: ((range includes: vmThread cStackPointer) and: [range includes: vmThread cFramePointer])]. - self setCFramePointer: vmThread cFramePointer setCStackPointer: vmThread cStackPointer. - self assert: newMethod notNil - ! Item was changed: ----- Method: CogThreadManager class>>initialize (in category 'class initialization') ----- initialize "CogThreadManager initialize" CTMUninitialized := 0. CTMInitializing := 1. CTMUnavailable := 2. "off doing its own thing and not available to run the VM, e.g. calling-out." CTMAssignableOrInVM := 3. "either owning the VM or blocked on its osSemaphore available for VM work" CTMWantingOwnership := 4. "with something specific to do in the VM (e.g. return a result from a call-out, make a call-back)" + + CTMUnknownOwner := -1. "Define the size of the stack of processes at time of disown/own." AWOLProcessesIncrement := 4! Item was changed: ----- Method: CogThreadManager>>acquireVMFor: (in category 'public api') ----- + acquireVMFor: vmThread - acquireVMFor: threadIndex "Attempt to acquire the VM, eventually blocking until it becomes available. Spin until the maxWaitingPriority has been updated if it is lower than this thread's priority." <returnTypeC: #'CogVMThread *'> - | vmThread | <var: #vmThread type: #'CogVMThread *'> + self assert: vmThread index = self ioGetThreadLocalThreadIndex. - self assert: threadIndex = self ioGetThreadLocalThreadIndex. - vmThread := self vmThreadAt: threadIndex. self assert: (vmThread vmThreadState = CTMUnavailable or: [vmThread vmThreadState = CTMWantingOwnership]). + (self tryLockVMOwnerTo: vmThread index) - (self tryLockVMOwnerTo: threadIndex) ifTrue: [vmThread setVmThreadState: CTMAssignableOrInVM] ifFalse: [vmThread setVmThreadState: CTMWantingOwnership. + [(self vmOwnerIs: vmThread index) + or: [self tryLockVMOwnerTo: vmThread index]] whileFalse: - [(self vmOwnerIs: threadIndex) - or: [self tryLockVMOwnerTo: threadIndex]] whileFalse: [vmThread priority ifNotNil: [coInterpreter waitingPriorityIsAtLeast: vmThread priority]. + (self vmOwnerIs: vmThread index) ifFalse: - (self vmOwnerIs: threadIndex) ifFalse: [self ioWaitOnOSSemaphore: (self addressOf: vmThread osSemaphore)]]]. coInterpreter assertProcessorStackPointersBelongToCurrentThread. vmOSThread := vmThread osThread. vmThread setVmThreadState: CTMAssignableOrInVM. ^vmThread! Item was changed: ----- Method: CogThreadManager>>assertValidProcessorStackPointersForIndex: (in category 'simulation') ----- assertValidProcessorStackPointersForIndex: threadIndex <cmacro: '(ignored) 0'> "simulation only" | time range | + threadIndex = CTMUnknownOwner + ifTrue: [^ self assertEmptyRegisterStates: cogit processor registerState]. time := Time utcMicrosecondClock. range := coInterpreter cStackRangeForThreadIndex: threadIndex. self assert: ((range includes: cogit processor fp) and: [range includes: cogit processor sp]) "(0 to: numThreads + numThreadsIncrement) detect: [:i| | range | range := coInterpreter cStackRangeForThreadIndex: threadIndex. ((range includes: cogit processor fp) and: [range includes: cogit processor sp])] ifNone: []" "{ coInterpreter whereIs: cogit processor fp. coInterpreter whereIs: cogit processor sp }"! Item was changed: ----- Method: CogThreadManager>>assertValidStackPointersInState:forIndex: (in category 'simulation') ----- assertValidStackPointersInState: registerState forIndex: threadIndex | time range | time := Time utcMicrosecondClock. range := coInterpreter cStackRangeForThreadIndex: threadIndex. self assert: ((range includes: (registerState at: cogit processor registerStateFPIndex)) and: [range includes: (registerState at: cogit processor registerStateSPIndex)]) "(1 to: self maxNumThreads) collect: [:i| | range | range := coInterpreter cStackRangeForThreadIndex: i. range storeStringBase: 16]" "(1 to: self maxNumThreads) detect: [:i| | range | range := coInterpreter cStackRangeForThreadIndex: i. ((range includes: (registerState at: cogit processor registerStateFPIndex)) and: [range includes: (registerState at: cogit processor registerStateSPIndex)])] ifNone: []" "{ (registerState at: cogit processor registerStateFPIndex) hex. coInterpreter whereIs: (registerState at: cogit processor registerStateFPIndex). (registerState at: cogit processor registerStateSPIndex) hex. coInterpreter whereIs: (registerState at: cogit processor registerStateSPIndex) }" "(1 to: self maxNumThreads) detect: [:i| | range | range := coInterpreter cStackRangeForThreadIndex: i. ((range includes: cogit processor fp) and: [range includes: cogit processor sp])] ifNone: []" "{ cogit processor fp hex. coInterpreter whereIs: cogit processor fp. cogit processor sp hex. coInterpreter whereIs: cogit processor sp }"! Item was changed: ----- Method: CogThreadManager>>doTryLockVMOwnerTo: (in category 'simulation') ----- doTryLockVMOwnerTo: threadIndex "In the simulation, this is being called by #simulateTryLockVMOwnerTo:, in C this method will just be called directly. Returns true if the vmOwner has been successfully set to the given thread index." <inline: #always> <var: #expected type: #int> | expected | expected := self cCode: 0 inSmalltalk: [AtomicValue newFrom: 0]. ^ (self atomic: (self addressOf: vmOwner) _compare: (self addressOf: expected) _exchange_strong: threadIndex) + or: ["We may already be vmOwner. The current vmOwner will be stored in expected. + However, if an unknown owner is present, we cannot assume that's us!!" + expected = threadIndex and: [threadIndex ~= CTMUnknownOwner]]! - or: ["We may already be vmOwner. The current vmOwner will be stored in expected" expected = threadIndex]! Item was changed: ----- Method: CogThreadManager>>ensureRunningVMThread: (in category 'public api') ----- ensureRunningVMThread: vmIsRelinquishing + "Called from checkVMOwnershipFromHeartbeat if the heartbeat thread manages to lock the vmOwner to CTMUnknownOwner. + Hence we are the vmOwner and are in the heartbeat thread. + We must therefore ensure that when we leave this method either: + 1. Some other thread is running and vmOwner + 2. We've released ownership of the VM" - "Called from checkVMOwnershipFromHeartbeat if the VM is unowned. - Hence we are in the heartbeat thread. The race is against that thread - owning the VM and against foreign callbacks." <returnTypeC: #void> <var: #vmThread type: #'CogVMThread *'> self willingVMThread ifNotNil: [:vmThread| | threadState | threadState := vmThread vmThreadState. "If the VM is relinquishing the processor then only schedule a thread if it has work to do." (vmIsRelinquishing and: [threadState ~= CTMWantingOwnership]) ifTrue: + [^self releaseVM]. - [^self]. self assert: ((threadState = CTMAssignableOrInVM or: [threadState = CTMInitializing]) or: [threadState = CTMWantingOwnership]). + "Ownership will be transferred to vmThread, no need to release the VM." + ^ self wakeVMThread: vmThread]. - (self tryLockVMOwnerTo: vmThread index) ifFalse: "someone beat us to it..." - [^self]. - vmOSThread := vmThread osThread. - "release the thread from its blocking loop" - self ioSignalOSSemaphore: (self addressOf: vmThread osSemaphore). - self ioTransferTimeslice. - "self cCode: [coInterpreter print: 'ERVT signalled '; printNum: vmThread index; cr]." - ^self]. "If the VM is relinquishing the processor then only schedule a thread if it has work to do (willingVMThread not nil above). If we have failed to allocate thread storage before there is no point continuing to try to do so. By this time we should have quite a few threads in the pool." (vmIsRelinquishing or: [memoryIsScarce]) ifTrue: + [^self releaseVM]. + self unusedThreadInfo ifNotNil: [:vmThread| ^ self wakeVMThread: vmThread]. + ^ self releaseVM! - [^self]. - self unusedThreadInfo ifNotNil: - [:vmThread| - (self tryLockVMOwnerTo: vmThread index) ifTrue: - [(self startThreadForThreadInfo: vmThread) ifFalse: - [self releaseVM]]]! Item was changed: ----- Method: CogThreadManager>>loadOrInitializeRegisterStateFor: (in category 'simulation') ----- loadOrInitializeRegisterStateFor: threadIndex <doNotGenerate> + |processor| + threadIndex = CTMUnknownOwner ifTrue: [^ self halt]. + - |fakeThreadIndex processor| - "The heartbeat thread will lock the VM to -1, so generate a fake processor data for this." - fakeThreadIndex := threadIndex == -1 ifTrue: [self maxNumThreads] ifFalse: [threadIndex]. processor := cogit processor. processor setRegisterState: (registerStates at: threadIndex ifAbsentPut: + [self initializeProcessor: processor forThreadIndex: threadIndex. - [self initializeProcessor: processor forThreadIndex: fakeThreadIndex. processor registerState]).! Item was changed: ----- Method: CogThreadManager>>saveRegisterStateForCurrentProcess (in category 'simulation') ----- saveRegisterStateForCurrentProcess "On switching osProcess we have to both - SAVE old process's register state - RESTORE new process's register state So what are the transitions? The transitions out (where processor state must be saved) are the opposites of tryLockVMOwnerTo:. Hence - waitOnOSSemaphore: - disownVM: - ioTransferTimeslice i.e. the continuations from here, disownVM:, and ioTransferTimeslice, will use tryLockVMOwnerTo: to regain control of the VM. So the register state to be restored at that point (if tryLockVMOwnerTo: succeeds) is the register state saved in one of the three places. But the processor is initialized in tryLockVMOwnerTo:, so only save the state if state is already present." <doNotGenerate> + self getVMOwner = CTMUnknownOwner ifTrue: [^ self]. cogit withProcessorHaltedDo: [| currentVMThread state | currentVMThread := self vmThreadForCurrentProcess. state := cogit processor registerState. self assertValidStackPointersInState: state forIndex: currentVMThread index. registerStates at: currentVMThread index put: state]! Item was changed: ----- Method: CogThreadManager>>simulateTryLockVMOwnerTo: (in category 'simulation') ----- simulateTryLockVMOwnerTo: threadIndex "In the real VM this is a direct call of #tryLockVMOwnerTo:. In the simulation this is where register state is restored, simulating a thread switch. State is stored in saveRegisterStateForCurrentProcess (sent by disownVM:, ioWaitOnOSSemaphore: and ioTransferTimeslice). The code here and in saveRegisterStateForCurrentProcess allow us to avoid the expensive and complex MultiProcessor hack. The idea here is to save the register state around the invocation of tryLockVMOwnerTo:, and set the register state to that for the owner, changing the state if ownership has changed, restoring it if ownership has not changed." <doNotGenerate> self deny: threadIndex = 0. ^cogit withProcessorHaltedDo: [| previousOwner currentOwner processor result | processor := cogit processor. "After switching, the 'current' owner will be the 'previous' owner. Though the value will be the same, let's still introduce a second variable that we can use after the switch to make it more clear what's going on." previousOwner := currentOwner := self getVMOwner. "If we currently have a VM owner, the register state should be + valid for that owner, otherwise it should be empty. + It may be CTMUnknownOwner (-1), in that case it should also be empty." + currentOwner > 0 - valid for that owner, otherwise it should be empty." - currentOwner ~= 0 ifTrue: [self assertValidStackPointersInState: processor registerState forIndex: currentOwner] ifFalse: [self assertEmptyRegisterStates: processor registerState]. result := self doTryLockVMOwnerTo: threadIndex. self assert: result = (threadIndex = self getVMOwner). result ifTrue: ["If we did actually change owners, assert that previously the processor was emtpy." previousOwner ~= self getVMOwner ifTrue: [self assertEmptyRegisterStates: processor registerState. self loadOrInitializeRegisterStateFor: threadIndex]]. coInterpreter transcript ensureCr; + f: 'tryLockVMOwner %d -> %d (%s) %s\n' + printf: { previousOwner. threadIndex. thisContext home sender selector. result ifTrue: ['ok'] ifFalse: ['FAILED'] }. - f: (result ifTrue: ['tryLockVMOwner %d -> %d (%s) ok\n'] ifFalse: ['tryLockVMOwner %d -> %d (%s) FAILED\n']) - printf: { previousOwner. threadIndex. thisContext home sender selector }. self assertValidProcessorStackPointersForIndex: self getVMOwner. result]! Item was added: + ----- Method: CogThreadManager>>wakeVMThread: (in category 'scheduling') ----- + wakeVMThread: vmThread + <var: #vmThread type: #'CogVMThread *'> + <returnTypeC: #void> + | threadState | + self assert: (self vmIsOwned and: [(self vmOwnerIs: vmThread index) not]). + + "Instead of going through a #disownVM: call, directly set the new VM owner. + This has the advantage of avoiding a race for the different threads to become the new + VM owner. + In Simulation, this means we need to simulate a thread-switch." + self cCode: [] inSmalltalk: [ + self saveRegisterStateForCurrentProcess. + self loadOrInitializeRegisterStateFor: vmThread index]. + self setVMOwner: vmThread index. + + threadState := vmThread vmThreadState. + threadState = CTMUninitialized + ifTrue: [(self startThreadForThreadInfo: vmThread) ifFalse: [self releaseVM. "TODO: IS THIS SANE?"]] + ifFalse: + [self assert: ((threadState = CTMWantingOwnership + or: [threadState = CTMAssignableOrInVM]) + or: [threadState = CTMInitializing]). + self ioSignalOSSemaphore: (self addressOf: vmThread osSemaphore)]. + self ioTransferTimeslice! Item was changed: ----- Method: CogThreadManager>>wakeVMThreadFor: (in category 'public api') ----- wakeVMThreadFor: index "Transfer the VM to the thread with index. Called from a thread that finds the highest priority runnable process is bound to the thread with index index." <returnTypeC: #void> - | vmThread threadState | - self assert: (self vmIsOwned and: [(self vmOwnerIs: index) not]). self assert: (index between: 1 and: numThreads). + ^ self wakeVMThread: (threads at: index). + ! - vmThread := threads at: index. - - "Instead of going through a #disownVM: call, directly set the new VM owner. - This has the advantage of avoiding a race for the different threads to become the new - VM owner. - In Simulation, this means we need to simulate a thread-switch." - self cCode: [] inSmalltalk: [ - self saveRegisterStateForCurrentProcess. - self loadOrInitializeRegisterStateFor: index]. - self setVMOwner: index. - - threadState := vmThread vmThreadState. - threadState = CTMUninitialized - ifTrue: [self startThreadForThreadInfo: vmThread] - ifFalse: - [self assert: ((threadState = CTMWantingOwnership - or: [threadState = CTMAssignableOrInVM]) - or: [threadState = CTMInitializing]). - self ioSignalOSSemaphore: (self addressOf: vmThread osSemaphore)]. - self ioTransferTimeslice! Item was changed: VMStructType subclass: #CogVMThread + instanceVariableNames: 'index state priority osSemaphore osThread disownFlags newMethodOrNull argumentCount inMachineCode cStackPointer cFramePointer awolProcIndex awolProcLength awolProcesses reenterThreadSchedulingLoop' - instanceVariableNames: 'index state priority osSemaphore osThread newMethodOrNull argumentCount inMachineCode cStackPointer cFramePointer awolProcIndex awolProcLength awolProcesses reenterThreadSchedulingLoop' classVariableNames: '' poolDictionaries: 'VMThreadingConstants' category: 'VMMaker-Multithreading'! !CogVMThread commentStamp: '<historical>' prior: 0! Instances of this class represent control blocks for native threads that cooperatively schedule the VM. See the class comment of CogThreadManager for full documentation. N.B. awolProcesses must be the last inst var.! Item was added: + ----- Method: CogVMThread>>awolProcesses: (in category 'accessing') ----- + awolProcesses: anObject + + ^ awolProcesses := anObject.! Item was added: + ----- Method: CogVMThread>>coerceTo:sim: (in category 'as yet unclassified') ----- + coerceTo: aSymbol sim: aSimulator + + self assert: aSymbol = #'CogVMThread *'. + ^ self! Item was added: + ----- Method: CogVMThread>>disownFlags (in category 'accessing') ----- + disownFlags + + ^ disownFlags! Item was added: + ----- Method: CogVMThread>>disownFlags: (in category 'accessing') ----- + disownFlags: anObject + + ^ disownFlags := anObject.! Item was changed: ----- Method: CogVMThread>>printOn: (in category 'printing') ----- printOn: aStream super printOn: aStream. aStream nextPutAll: ' index: '; print: index; nextPutAll: ' state: '; nextPutAll: (VMThreadingConstants keys + detect: [:k| k first == $C and: [(VMThreadingConstants classPool at: k) = state value]] + ifNone: [state value printString])! - detect: [:k| k first == $C and: [(VMThreadingConstants classPool at: k) = state]] - ifNone: [state printString])! Item was changed: ----- Method: FilePlugin>>primitiveFileReadPinningAndDisowning (in category 'file primitives') ----- primitiveFileReadPinningAndDisowning "This version of primitiveFileRead is for garbage collectors that support pinning and the multi-threaded VM. It actually does the own/disown dance." + | count startIndex array file slotSize elementSize bytesRead vmHandle wasPinned | - | count startIndex array file slotSize elementSize bytesRead threadIndexAndFlags wasPinned | <inline: true> <var: 'file' type: #'SQFile *'> <var: 'count' type: #'size_t'> <var: 'startIndex' type: #'size_t'> <var: 'slotSize' type: #'size_t'> <var: 'elementSize' type: #'size_t'> count := interpreterProxy positiveMachineIntegerValueOf: (interpreterProxy stackValue: 0). startIndex := interpreterProxy positiveMachineIntegerValueOf: (interpreterProxy stackValue: 1). array := interpreterProxy stackValue: 2. file := self fileValueOf: (interpreterProxy stackValue: 3). (interpreterProxy failed "buffer can be any indexable words or bytes object except CompiledMethod" or: [(interpreterProxy isWordsOrBytes: array) not]) ifTrue: [^interpreterProxy primitiveFailFor: PrimErrBadArgument]. slotSize := interpreterProxy slotSizeOf: array. (startIndex >= 1 and: [startIndex + count - 1 <= slotSize]) ifFalse: [^interpreterProxy primitiveFailFor: PrimErrBadIndex]. (wasPinned := interpreterProxy isPinned: array) ifFalse: [array := interpreterProxy pinObject: array]. + vmHandle := interpreterProxy disownVM: DisownVMForThreading. - threadIndexAndFlags := interpreterProxy disownVM: DisownVMForThreading. "Note: adjust startIndex for zero-origin byte indexing" elementSize := slotSize = 0 ifTrue: [1] ifFalse: [(interpreterProxy byteSizeOf: array) // slotSize]. bytesRead := self sqFile: file Read: count * elementSize Into: (interpreterProxy cCoerce: (interpreterProxy firstIndexableField: array) to: #'char *') At: startIndex - 1 * elementSize. + interpreterProxy ownVM: vmHandle. - interpreterProxy ownVM: threadIndexAndFlags. wasPinned ifFalse: [interpreterProxy unpinObject: array]. interpreterProxy failed ifFalse: [interpreterProxy methodReturnInteger: bytesRead // elementSize] "answer # of elements read"! Item was changed: ----- Method: TMethod>>inlineSend:directReturn:exitVar:in: (in category 'inlining') ----- inlineSend: aSendNode directReturn: directReturn exitVar: exitVar in: aCodeGen "Answer a collection of statements to replace the given send. directReturn indicates that the send is the expression in a return statement, so returns can be left in the body of the inlined method. If exitVar is nil, the value returned by the send is not used; thus, returns need not assign to the output variable. Types are propagated to as-yet-untyped variables when inlining a send that is assigned, otherwise the assignee variable type must match the return type of the inlinee. Return types are not propagated." | sel meth methArgs exitLabel inlineStmts label exitType elidedArgs | sel := aSendNode selector. meth := aCodeGen methodNamed: sel. methArgs := meth args. "convenient for debugging..." aCodeGen maybeBreakForInlineOf: aSendNode in: self. elidedArgs := #(). (methArgs notEmpty and: [methArgs first beginsWith: 'self_in_']) ifTrue: "If the first arg is not used we can and should elide it." [| varNode | varNode := TVariableNode new setName: methArgs first. (meth parseTree noneSatisfy: [:node| varNode isSameAs: node]) ifTrue: [elidedArgs := {methArgs first}]. methArgs := methArgs allButFirst]. methArgs size = aSendNode args size ifFalse: [^nil]. meth := meth copy. (meth statements size > 1 and: [meth statements first isSend and: [meth statements first selector == #flag:]]) ifTrue: [meth statements removeFirst]. "Propagate the return type of an inlined method" (directReturn or: [exitVar notNil]) ifTrue: [exitType := directReturn ifTrue: [returnType] ifFalse: [(self typeFor: exitVar in: aCodeGen) ifNil: [#sqInt]]. (exitType = #void or: [exitType = meth returnType]) ifFalse: [meth propagateReturnIn: aCodeGen]]. "Propagate any unusual argument types to untyped argument variables" methArgs with: aSendNode args do: [:formal :actual| (meth declarationAt: formal ifAbsent: nil) ifNil: [(self typeFor: actual in: aCodeGen) ifNotNil: [:type| type ~= #sqInt ifTrue: [meth declarationAt: formal put: (type last = $* ifTrue: [type, formal] ifFalse: [type, ' ', formal])]]]]. meth renameVarsForInliningInto: self except: elidedArgs in: aCodeGen. meth renameLabelsForInliningInto: self. self addVarsDeclarationsAndLabelsOf: meth except: elidedArgs. meth hasReturn ifTrue: [directReturn ifFalse: [exitLabel := self unusedLabelForInliningInto: self. (meth exitVar: exitVar label: exitLabel) "is label used?" ifTrue: [ labels add: exitLabel ] ifFalse: [ exitLabel := nil ]]]. (inlineStmts := OrderedCollection new: meth statements size + meth args size + 2) add: (label := TLabeledCommentNode new setComment: 'begin ', sel); addAll: (self argAssignmentsFor: meth send: aSendNode except: elidedArgs in: aCodeGen); addAll: meth statements. "method body" directReturn ifTrue: [meth endsWithReturn ifTrue: [exitVar ifNotNil: "don't remove the returns if being invoked in the context of a return" [inlineStmts at: inlineStmts size put: inlineStmts last copyWithoutReturn]] ifFalse: [inlineStmts add: (TReturnNode new setExpression: (TVariableNode new setName: 'nil'))]]. exitLabel ifNotNil: [inlineStmts add: (TLabeledCommentNode new setLabel: exitLabel comment: 'end ', meth selector)]. inlineStmts size = 1 ifTrue: "Nuke empty methods; e.g. override of flushAtCache" [self assert: inlineStmts first isComment. inlineStmts removeFirst]. ^inlineStmts! Item was changed: ----- Method: TMethod>>transformToStructClassMethodFor: (in category 'transformations') ----- transformToStructClassMethodFor: aCCodeGenerator "Transform this method so that it can be used on an instance of a struct class (VMStructType subclass). Convert inst var refs into field dereferences of self. Add selfSelector as the first argument with the right struct type. As a complete hack to avoid breaking the inlinert don't use 'self' as the name for self as this causes serious type redefinitions ``somewhere'' in the inliner." | replacements selfNode typeForSelf | self isStructAccessor ifTrue: [^self returnType: (definingClass returnTypeForAccessor: selector)]. replacements := IdentityDictionary new. selfNode := TVariableNode new setName: 'self_in_', (aCCodeGenerator cFunctionNameFor: selector). args do: [:var| (definingClass isAccessor: var) ifTrue: [self error: 'In ', definingClass name, '>>', selector, ' ', var, ' arg shadows struct field and will break during translation!!']]. parseTree nodesDo: [:node| node isVariable ifTrue: [node name = 'self' ifTrue: [replacements at: node put: selfNode copy]. (definingClass isAccessor: node name) ifTrue: [replacements at: node put: (TSendNode new setSelector: node name asSymbol receiver: selfNode arguments: #())]]]. replacements notEmpty ifTrue: [self replaceNodesIn: replacements]. typeForSelf := self typeForSelf. self assert: (typeForSelf notNil and: [typeForSelf ~~ #implicit]). self declarationAt: (args addFirst: selfNode name) put: (declarations removeKey: 'self'), '_in_', (aCCodeGenerator cFunctionNameFor: selector)! Item was changed: ----- Method: TSendNode>>emitCCodeAsArgumentOn:level:generator: (in category 'C code generation') ----- emitCCodeAsArgumentOn: aStream level: level generator: aCodeGen "Emit the receiver in a form that can be passed as an argument." "If the selector is a built-in construct, translate it and return" (aCodeGen emitBuiltinConstructAsArgumentFor: self on: aStream level: level) ifFalse: ["If it is a pointer dereference generate it" (self emitCCodeAsFieldReferenceOn: aStream level: level generator: aCodeGen) ifFalse: ["Otherwise generate the vanilla C function call." self emitCCodeAsFunctionCallOn: aStream level: level generator: aCodeGen]]! Item was changed: ----- Method: TSendNode>>emitCCodeAsFieldReferenceOn:level:generator: (in category 'C code generation') ----- emitCCodeAsFieldReferenceOn: aStream level: level generator: aCodeGen "If appropriate, translate this message send as a pointer dereference" | parenCount | (aCodeGen isStructSend: self) ifFalse: [^false]. (aCodeGen shouldGenerateAsInterpreterProxySend: self) ifTrue: [^false]. + "(selector = #state + and: [(receiver typeOrNilFrom: aCodeGen in: aCodeGen currentMethod) = 'CogVMThread *']) + ifTrue: [(CogVMThread>>#state asTranslationMethodOfClass: TMethod) + emitInlineOn: aStream level: level generator: aCodeGen. + ^ true + ^ false.]." parenCount := receiver isSend ifTrue: [2] ifFalse: [1]. aStream next: parenCount put: $(. receiver emitCCodeAsExpressionOn: aStream level: 0 generator: aCodeGen. parenCount > 1 ifTrue: [aStream nextPut: $)]. (receiver structTargetKindIn: aCodeGen) caseOf: { [#pointer] -> [aStream nextPut: $-; nextPut: $>]. [#struct] -> [aStream nextPut: $.] }. aStream nextPutAll: (aCodeGen cFunctionNameFor: selector). arguments isEmpty ifFalse: [self assert: arguments size = 1. aStream nextPutAll: ' = '. arguments first emitCCodeAsExpressionOn: aStream level: level generator: aCodeGen]. aStream nextPut: $). ^true! Item was changed: ----- Method: TSendNode>>emitCCodeAsFunctionCallOn:level:generator: (in category 'C code generation') ----- emitCCodeAsFunctionCallOn: aStream level: level generator: aCodeGen "Translate this message send into a C function call" selector == #break ifTrue: [aStream nextPutAll: '/* send of break elided */'. ^self]. "Special case for pluggable modules. Replace messages to interpreterProxy by interpreterProxy->message(..) if the message is not builtin" (aCodeGen shouldGenerateAsInterpreterProxySend: self) ifTrue: [(aCodeGen noteUsedPluginFunction: selector) ifTrue: [aStream nextPutAll: 'interpreterProxy->']]. aStream nextPutAll: (aCodeGen cFunctionNameFor: selector); nextPut: $(. "Only include the receiver as the first argument in certain cases. The receiver is always included if it is an expression. If it is a variable: If the vmClass says it is an implicit variable, don't include it. If the variable is 'self' and the method being called is not in the method set (i.e. it is some external code), don't include it." (self shouldExcludeReceiverAsFirstArgument: aCodeGen) ifFalse: [(receiver structTargetKindIn: aCodeGen) == #struct ifTrue: [aStream nextPut: $&]. "Hack fix warnings for isCMOpenPIC, isCMClosedPIC, et al" (self shouldCastReceiverIfStructIn: aCodeGen ifTrue: [:formalParameterType| aCodeGen nodeToCast: receiver to: formalParameterType] ifFalse: [receiver]) emitCCodeOn: aStream level: level generator: aCodeGen. arguments isEmpty ifFalse: [aStream nextPutAll: ', ']]. arguments do: [ :arg| arg emitCCodeAsArgumentOn: aStream level: level generator: aCodeGen] separatedBy: (self argumentSeparationBlockFor: aStream numArgs: arguments size level: level). aStream nextPut: $)! Item was changed: ----- Method: TSendNode>>emitCCodeOn:level:generator: (in category 'C code generation') ----- emitCCodeOn: aStream level: level generator: aCodeGen "Emit the receiver as a statement." "If the selector is a built-in construct, translate it and return" (aCodeGen emitBuiltinConstructFor: self on: aStream level: level) ifFalse: ["If it is a pointer dereference generate it" (self emitCCodeAsFieldReferenceOn: aStream level: level generator: aCodeGen) ifFalse: ["Otherwise generate the vanilla C function call." self emitCCodeAsFunctionCallOn: aStream level: level generator: aCodeGen]]! Item was changed: SharedPool subclass: #VMThreadingConstants instanceVariableNames: '' + classVariableNames: 'AWOLProcessesIncrement CTMAssignableOrInVM CTMInitializing CTMUnavailable CTMUninitialized CTMUnknownOwner CTMWantingOwnership ThreadIdIndex ThreadIdShift' - classVariableNames: 'AWOLProcessesIncrement CTMAssignableOrInVM CTMInitializing CTMUnavailable CTMUninitialized CTMWantingOwnership ThreadIdIndex ThreadIdShift' poolDictionaries: '' category: 'VMMaker-Multithreading'! !VMThreadingConstants commentStamp: '<historical>' prior: 0! VMThreadingConstants ensureClassPool. CogThreadManager classPool keys do: [:k| VMThreadingConstants classPool declare: k from: CogThreadManager classPool]. CoInterpreterMT classPool keys do: [:k| VMThreadingConstants classPool declare: k from: CoInterpreterMT classPool].!

1 0

Building VMMaker image supporting AMD64 and AArch64 ISAs on Linux
by Jan Vrany 04 Oct '23

04 Oct '23

Hi Eliot and others, I'm trying to build working VMMaker image in order to see what machine code Cog generates for some simple methods - without success so far. All this is on Linux on AMD64. Detailed report on what exactly I did and what errors I got is below. In short: (i) does AArch64 sim plugin works on modern linuxes, does anyone use it there? (ii) if so, what's the correct incantation to compile it. (iii) using code in workspace labelled "In-Image Compilation Workspace", is it supposed to work for every method and every ISA or is it normal that it fails with DNU for some methods? I managed to compile but the trivial ones. Thanks! Jan ================================================== ==== Below details what I did and errors I got === I'm using this commit: commit 330d6779ad2ecbce1f07131f53d75cd168165f0b (HEAD -> Cog, origin/HEAD, origin/Cog) Author: Christoph Thiede <christoph.thiede(a)student.hpi.de> Date: Thu Aug 17 13:32:52 2023 +0200 Here's what I did: * IIUC, first I need to build bochs and BOCHS 64 simulator, so cd opensmalltalk-vm/building/linux64x64/bochsx64 ./conf.COG ./makeem That built just fine. * Then I built gdbarm64 simulator: cd opensmalltalk-vm/building/linux64x64/gdbarm64$ ./conf.COG ./makeem Again, no problem so far. * Then I tried to build the VM: cd opensmalltalk-vm/building/linux64x64/squeak.cog.spur/build ./mvm That failed soon: .../opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:1832:10: warning: format string is not a string literal (potentially insecure) [-Wformat-security] printf(optionStrings[count]); ^~~~~~~~~~~~~~~~~~~~ .../opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:1832:10: note: treat the string as an argument to avoid this printf(optionStrings[count]); ^ "%s", .../opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:2324:38: error: incompatible function pointer types assigning to 'void (*)(int, siginfo_t *, void *)' from 'void (int, siginfo_t *, ucontext_t *)' (aka 'void (int, siginfo_t *, struct ucontext_t *)') [-Wincompatible-function-pointer-types] sigsegv_handler_action.sa_sigaction = sigsegv; ^ ~~~~~~~ .../opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:2331:38: error: incompatible function pointer types assigning to 'void (*)(int, siginfo_t *, void *)' from 'void (int, siginfo_t *, ucontext_t *)' (aka 'void (int, siginfo_t *, struct ucontext_t *)') [-Wincompatible-function-pointer-types] sigusr1_handler_action.sa_sigaction = sigusr1; ^ ~~~~~~~ 1 warning and 2 errors generated. This is using clang version 16.0.6 (which mvm script enforces on line 20) * Then I tried to build the VM again, this time with GCC (version 13.2.0 by editing mvm line 20): cd opensmalltalk-vm/building/linux64x64/squeak.cog.spur/build ./mvm This failed to compile too, though with different error: libtool: compile: gcc -Wall -g -O2 -DNDEBUG -DDEBUGVM=0 -msse2 -DCOGMTVM=0 -pthread -DLSB_FIRST=1 -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused- value -Wno-unused-label -Wno-unused-function -Wno-unused-variable -Wno-unused-but-set-variable -DHAVE_CONFIG_H -I.../opensmalltalk- vm/building/linux64x64/squeak.cog.spur/build -I.../opensmalltalk-vm/building/linux64x64/squeak.cog.spur/build -I.../opensmalltalk-vm/platforms/unix/vm - I.../opensmalltalk-vm/platforms/Cross/vm -I.../opensmalltalk-vm/src/spur64.cog -I/usr/local/include -I.../opensmalltalk- vm/platforms/Cross/plugins/GdbARMv8Plugin -I../../../../processors/ARM/gdb-8.3.1/include -I../../../../processors/ARM/gdb-8.3.1/sim/aarch64 -m64 -Wno-missing- braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable -Wno-unused-but-set-variable -c .../opensmalltalk- vm/platforms/Cross/plugins/GdbARMv8Plugin/sqGdbARMv8Plugin.c -fPIC -DPIC -o .libs/sqGdbARMv8Plugin.o In file included from .../opensmalltalk-vm/platforms/Cross/vm/sqMemoryAccess.h:23, from .../opensmalltalk-vm/platforms/unix/vm/sqPlatformSpecific.h:35, from .../opensmalltalk-vm/platforms/Cross/vm/sqAssert.h:16, from .../opensmalltalk-vm/platforms/Cross/plugins/GdbARMv8Plugin/sqGdbARMv8Plugin.c:4: .../opensmalltalk-vm/building/linux64x64/squeak.cog.spur/build/config.h:40:2: error: #error This file was included too late. Please make sure it is included earlier 40 | #error This file was included too late. Please make sure it is included earlier | ^~~~~ .../opensmalltalk-vm/platforms/Cross/plugins/GdbARMv8Plugin/sqGdbARMv8Plugin.c:9:10: fatal error: aarch64/config.h: No such file or directory 9 | #include <aarch64/config.h> | ^~~~~~~~~~~~~~~~~~ compilation terminated. make[1]: *** [Makefile:197: sqGdbARMv8Plugin.lo] Error 1 make: *** [Makefile:557: GdbARMv8Plugin.la] Error 2 * Then I gave up on AArch64 support and tried only with AMD64 by cleaning opensmalltalk-vm/building/linux64x64/gdbarm64. This time VM compiled fine. * As a next step, I built VMMaker image: cd opensmalltalk-vm/image ./buildspurtrunkvmmaker64image.sh -vm ../building/linux64x64/squeak.cog.spur/build/squeak.sh This worked just fine, producing Spur64VMMaker.image * Finally, I tried to see AMD64 (since I could not build AArch64 sim plugin) machine code for some method, so I executed following smalltalk code in a workspace (which is just a modification of what is suggested in one of pre-opened workspaces I only changed memory manager class and ISA): StackToRegisterMappingCogit genAndDis: SequenceableCollection>>#indexOf:startingAt:ifAbsent: options: #(ObjectMemory Spur64BitCoMemoryManager ISA X64). This resulted in DNU: 28 September 2023 1:07:59.435358 pm VM: unix - Smalltalk Image: Squeak6.1alpha [latest update: #22775] SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir /home/jv/Projects/Squeak/opensmalltalk-vm/image Trusted Dir /home/jv/Projects/Squeak/opensmalltalk-vm/image/secure Untrusted Dir /home/jv/Projects/Squeak/opensmalltalk-vm/image/My Squeak UndefinedObject(Object)>>doesNotUnderstand: #isCMBlock Receiver: nil Arguments and temporary variables: aMessage: isCMBlock exception: MessageNotUnderstood: UndefinedObject>>isCMBlock resumeValue: nil Receiver's instance variables: nil StackToRegisterMappingCogit(Cogit)>>disassembleMethod:on: Receiver: a StackToRegisterMappingCogit Arguments and temporary variables: surrogateOrAddress: nil aStream: a TranscriptStream cogMethod: nil mapEntries: nil codeRanges: nil i: nil iLimiT: nil Receiver's instance variables: coInterpreter: a CurrentImageCoInterpreterFacadeFor64BitSpurObjectRepresentation...etc... objectMemory: a CurrentImageCoInterpreterFacadeFor64BitSpurObjectRepresentation...etc... objectRepresentation: a CogObjectRepresentationFor64BitSpur processor: a BochsX64Alien64 rax: 00000101 (257) rbx: 00000000 rcx: 00000000 rdx...etc... processorLock: a Mutex() methodZone: a CogMethodZone methodZoneBase: 5784 codeBase: 1024 lastNInstructions: an OrderedCollection(a DoubleWordArray(0 0 0 0 4193216 41932...etc... simulatedAddresses: a Dictionary(#argumentCount->9223372036854775932 #ceActivateFailingPrimitiveMethod:...etc... simulatedTrampolines: a Dictionary(9223372036854775980->MessageSend(#ceSend:super:to:numArgs:...etc... simulatedVariableGetters: a Dictionary() simulatedVariableSetters: a Dictionary() printRegisters: false printInstructions: false compilationTrace: 0 clickConfirm: false clickStepping: false breakPC: nil breakBlock: nil singleStep: true guardPageSize: 1024 traceFlags: 8 traceStores: 0 breakMethod: nil methodObj: nil enumeratingCogMethod: nil methodHeader: nil initialPC: 0 endPC: nil methodOrBlockNumArgs: 0 inBlock: nil needsFrame: nil hasYoungReferent: false hasMovableLiteral: true primitiveIndex: nil backEnd: a CogInLineLiteralsX64Compiler literalsManager: an InLineLiteralsManager methodLabel: a CogInLineLiteralsX64Compiler (Label 0 @1698) stackCheckLabel: nil blockEntryLabel: nil blockEntryNoContextSwitch: nil blockNoContextSwitchOffset: nil stackOverflowCall: a CogInLineLiteralsX64Compiler (MoveCqR 0 ReceiverResultReg ...etc... sendMiss: a CogInLineLiteralsX64Compiler (Call 1680/690 E8 A0 F0 FF FF @15EB) missOffset: 48 entryPointMask: 7 checkedEntryAlignment: 0 uncheckedEntryAlignment: 7 cmEntryOffset: 48 entry: a CogInLineLiteralsX64Compiler (Label 1 @16C8) cmNoCheckEntryOffset: 71 noCheckEntry: a CogInLineLiteralsX64Compiler (Label 3 @1607) fullBlockEntry: nil cbEntryOffset: nil fullBlockNoContextSwitchEntry: nil cbNoSwitchEntryOffset: nil picMNUAbort: a CogInLineLiteralsX64Compiler (MoveCqR 0 ClassReg 48 31 C9 @16C0)...etc... picInterpretAbort: a CogInLineLiteralsX64Compiler (Call 2336/920 E8 58 F2 FF FF...etc... endCPICCase0: a CogInLineLiteralsX64Compiler (Label 3 @1617) endCPICCase1: a CogInLineLiteralsX64Compiler (Label 4 @162D) firstCPICCaseOffset: 87 cPICCaseSize: 22 cPICEndSize: 12 closedPICSize: 216 openPICSize: 360 fixups: a CArrayAccessor on: {a CogSSBytecodeFixup ( 0 ix 2) . a CogSSBytecodeFixup...etc... abstractOpcodes: a CArrayAccessor on: {a CogInLineLiteralsX64Compiler (MoveCqR ...etc... generatorTable: a CArrayAccessor on: {a CogBytecodeDescriptor (genPushReceiverVariableBytecode...etc... byte0: nil byte1: nil byte2: nil byte3: nil bytecodePC: nil bytecodeSetOffset: nil opcodeIndex: 64 numAbstractOpcodes: 100 blockStarts: nil blockCount: nil labelCounter: 5 cStackAlignment: 32 expectedSPAlignment: 0 expectedFPAlignment: 16 codeModified: nil maxLitIndex: nil ceMethodAbortTrampoline: nil cePICAbortTrampoline: nil ceCheckForInterruptTrampoline: 4648 ceCPICMissTrampoline: nil ceReturnToInterpreterTrampoline: 4872 ceBaseFrameReturnTrampoline: 4920 ceReapAndResetErrorCodeTrampoline: 2768 ceSendMustBeBooleanAddTrueTrampoline: 4544 ceSendMustBeBooleanAddFalseTrampoline: 4488 ceCannotResumeTrampoline: 4800 ceEnterCogCodePopReceiverReg: 4960 ceCallCogCodePopReceiverReg: 4984 ceCallCogCodePopReceiverAndClassRegs: 5008 cePrimReturnEnterCogCode: 5032 cePrimReturnEnterCogCodeProfiling: 5120 ceNonLocalReturnTrampoline: 4600 ceFetchContextInstVarTrampoline: 4696 ceStoreContextInstVarTrampoline: 4744 ceEnclosingObjectTrampoline: nil ceFlushDCache: nil ceFlushICache: nil ceTraceLinkedSendTrampoline: 9223372036854776172 ceTraceBlockActivationTrampoline: 9223372036854776180 ceTraceStoreTrampoline: 9223372036854776188 ceGetFP: 1024 ceGetSP: 1032 ceCaptureCStackPointers: 1040 ceInvokeInterpret: 4840 ordinarySendTrampolines: a CArrayAccessor on: #(1144 1208 1280 1352) superSendTrampolines: a CArrayAccessor on: #(1408 1480 1552 1624) directedSuperSendTrampolines: nil directedSuperBindingSendTrampolines: nil dynamicSuperSendTrampolines: nil outerSendTrampolines: nil selfSendTrampolines: nil firstSend: 1144 lastSend: 1624 realCEEnterCogCodePopReceiverReg: nil realCECallCogCodePopReceiverReg: nil realCECallCogCodePopReceiverAndClassRegs: nil trampolineTableIndex: 124 trampolineAddresses: a CArrayAccessor on: #('ceGetFP' 1024 'ceGetSP' 1032 'ceCa...etc... objectReferencesInRuntime: a CArrayAccessor on: #() runtimeObjectRefIndex: 0 cFramePointerInUse: true ceTryLockVMOwner: nil extA: 0 extB: 0 numExtB: 0 tempOop: nil numIRCs: nil indexOfIRC: nil theIRCs: nil receiverTags: nil implicitReceiverSendTrampolines: nil cogMethodSurrogateClass: CogMethodSurrogate64 cogBlockMethodSurrogateClass: CogBlockMethodSurrogate64 nsSendCacheSurrogateClass: NSSendCacheSurrogate64 cPICPrototype: a CogMethodSurrogate64 @5568/16r15C0 cPICEndOfCodeOffset: 209 cPICEndOfCodeLabel: a CogInLineLiteralsX64Compiler (Label 6 @1691) ceMallocTrampoline: nil ceFreeTrampoline: nil ceFFICalloutTrampoline: nil debugBytecodePointers: a Set() debugOpcodeIndices: a Set() disassemblingMethod: nil cogConstituentIndex: nil directedSendUsesBinding: nil processorFrameValid: false codeToDataDelta: 0 varBaseAddress: 2097160 codeZoneIsExecutableNotWritable: false debugAPISelector: nil shortCutTrampolineBlocks: an IdentityDictionary(#ceShortCutTraceBlockActivation:...etc... ioHighResClock: 996 perMethodProfile: nil instructionProfile: nil primitiveGeneratorTable: a CArrayAccessor on: {a CogPrimitiveDescriptor . a CogPrimitiveDescriptor...etc... introspectionDataIndex: nil introspectionData: nil prevBCDescriptor: nil numPushNilsFunction: #v3:Num:Push:Nils: pushNilSizeFunction: #v3PushNilSize:numInitialNils: methodOrBlockNumTemps: nil regArgsHaveBeenPushed: nil simStack: a CArrayAccessor on: {a CogSimStackEntry . a CogSimStackEntry . a CogSimStackEntry...etc... simStackPtr: nil simSpillBase: nil ceCallCogCodePopReceiverArg0Regs: 5248 ceCallCogCodePopReceiverArg1Arg0Regs: 5272 methodAbortTrampolines: a CArrayAccessor on: #(1680 1792 1904 2016) picAbortTrampolines: a CArrayAccessor on: #(2104 2216 2336 2456) picMissTrampolines: a CArrayAccessor on: #(2552 2608 2664 2720) ceCall0ArgsPIC: 5296 ceCall1ArgsPIC: 5328 ceCall2ArgsPIC: 5360 debugStackPointers: nil debugFixupBreaks: a Set() realCECallCogCodePopReceiverArg0Regs: nil realCECallCogCodePopReceiverArg1Arg0Regs: nil deadCode: nil useTwoPaths: nil currentCallCleanUpSize: nil simNativeStack: a CArrayAccessor on: {a CogSimStackNativeEntry . a CogSimStackNativeEntry...etc... simNativeStackPtr: nil simNativeSpillBase: nil simNativeStackSize: nil hasNativeFrame: nil compilationPass: nil StackToRegisterMappingCogit(Cogit)>>disassembleMethod: Receiver: a StackToRegisterMappingCogit Arguments and temporary variables: surrogateOrAddress: nil Receiver's instance variables: coInterpreter: a CurrentImageCoInterpreterFacadeFor64BitSpurObjectRepresentation...etc... objectMemory: a CurrentImageCoInterpreterFacadeFor64BitSpurObjectRepresentation...etc... objectRepresentation: a CogObjectRepresentationFor64BitSpur processor: a BochsX64Alien64 rax: 00000101 (257) rbx: 00000000 rcx: 00000000 rdx...etc... processorLock: a Mutex() methodZone: a CogMethodZone methodZoneBase: 5784 codeBase: 1024 lastNInstructions: an OrderedCollection(a DoubleWordArray(0 0 0 0 4193216 41932...etc... simulatedAddresses: a Dictionary(#argumentCount->9223372036854775932 #ceActivateFailingPrimitiveMethod:...etc... simulatedTrampolines: a Dictionary(9223372036854775980->MessageSend(#ceSend:super:to:numArgs:...etc... simulatedVariableGetters: a Dictionary() simulatedVariableSetters: a Dictionary() printRegisters: false printInstructions: false compilationTrace: 0 clickConfirm: false clickStepping: false breakPC: nil breakBlock: nil singleStep: true guardPageSize: 1024 traceFlags: 8 traceStores: 0 breakMethod: nil methodObj: nil enumeratingCogMethod: nil methodHeader: nil initialPC: 0 endPC: nil methodOrBlockNumArgs: 0 inBlock: nil needsFrame: nil hasYoungReferent: false hasMovableLiteral: true primitiveIndex: nil backEnd: a CogInLineLiteralsX64Compiler literalsManager: an InLineLiteralsManager methodLabel: a CogInLineLiteralsX64Compiler (Label 0 @1698) stackCheckLabel: nil blockEntryLabel: nil blockEntryNoContextSwitch: nil blockNoContextSwitchOffset: nil stackOverflowCall: a CogInLineLiteralsX64Compiler (MoveCqR 0 ReceiverResultReg ...etc... sendMiss: a CogInLineLiteralsX64Compiler (Call 1680/690 E8 A0 F0 FF FF @15EB) missOffset: 48 entryPointMask: 7 checkedEntryAlignment: 0 uncheckedEntryAlignment: 7 cmEntryOffset: 48 entry: a CogInLineLiteralsX64Compiler (Label 1 @16C8) cmNoCheckEntryOffset: 71 noCheckEntry: a CogInLineLiteralsX64Compiler (Label 3 @1607) fullBlockEntry: nil cbEntryOffset: nil fullBlockNoContextSwitchEntry: nil cbNoSwitchEntryOffset: nil picMNUAbort: a CogInLineLiteralsX64Compiler (MoveCqR 0 ClassReg 48 31 C9 @16C0)...etc... picInterpretAbort: a CogInLineLiteralsX64Compiler (Call 2336/920 E8 58 F2 FF FF...etc... endCPICCase0: a CogInLineLiteralsX64Compiler (Label 3 @1617) endCPICCase1: a CogInLineLiteralsX64Compiler (Label 4 @162D) firstCPICCaseOffset: 87 cPICCaseSize: 22 cPICEndSize: 12 closedPICSize: 216 openPICSize: 360 fixups: a CArrayAccessor on: {a CogSSBytecodeFixup ( 0 ix 2) . a CogSSBytecodeFixup...etc... abstractOpcodes: a CArrayAccessor on: {a CogInLineLiteralsX64Compiler (MoveCqR ...etc... generatorTable: a CArrayAccessor on: {a CogBytecodeDescriptor (genPushReceiverVariableBytecode...etc... byte0: nil byte1: nil byte2: nil byte3: nil bytecodePC: nil bytecodeSetOffset: nil opcodeIndex: 64 numAbstractOpcodes: 100 blockStarts: nil blockCount: nil labelCounter: 5 cStackAlignment: 32 expectedSPAlignment: 0 expectedFPAlignment: 16 codeModified: nil maxLitIndex: nil ceMethodAbortTrampoline: nil cePICAbortTrampoline: nil ceCheckForInterruptTrampoline: 4648 ceCPICMissTrampoline: nil ceReturnToInterpreterTrampoline: 4872 ceBaseFrameReturnTrampoline: 4920 ceReapAndResetErrorCodeTrampoline: 2768 ceSendMustBeBooleanAddTrueTrampoline: 4544 ceSendMustBeBooleanAddFalseTrampoline: 4488 ceCannotResumeTrampoline: 4800 ceEnterCogCodePopReceiverReg: 4960 ceCallCogCodePopReceiverReg: 4984 ceCallCogCodePopReceiverAndClassRegs: 5008 cePrimReturnEnterCogCode: 5032 cePrimReturnEnterCogCodeProfiling: 5120 ceNonLocalReturnTrampoline: 4600 ceFetchContextInstVarTrampoline: 4696 ceStoreContextInstVarTrampoline: 4744 ceEnclosingObjectTrampoline: nil ceFlushDCache: nil ceFlushICache: nil ceTraceLinkedSendTrampoline: 9223372036854776172 ceTraceBlockActivationTrampoline: 9223372036854776180 ceTraceStoreTrampoline: 9223372036854776188 ceGetFP: 1024 ceGetSP: 1032 ceCaptureCStackPointers: 1040 ceInvokeInterpret: 4840 ordinarySendTrampolines: a CArrayAccessor on: #(1144 1208 1280 1352) superSendTrampolines: a CArrayAccessor on: #(1408 1480 1552 1624) directedSuperSendTrampolines: nil directedSuperBindingSendTrampolines: nil dynamicSuperSendTrampolines: nil outerSendTrampolines: nil selfSendTrampolines: nil firstSend: 1144 lastSend: 1624 realCEEnterCogCodePopReceiverReg: nil realCECallCogCodePopReceiverReg: nil realCECallCogCodePopReceiverAndClassRegs: nil trampolineTableIndex: 124 trampolineAddresses: a CArrayAccessor on: #('ceGetFP' 1024 'ceGetSP' 1032 'ceCa...etc... objectReferencesInRuntime: a CArrayAccessor on: #() runtimeObjectRefIndex: 0 cFramePointerInUse: true ceTryLockVMOwner: nil extA: 0 extB: 0 numExtB: 0 tempOop: nil numIRCs: nil indexOfIRC: nil theIRCs: nil receiverTags: nil implicitReceiverSendTrampolines: nil cogMethodSurrogateClass: CogMethodSurrogate64 cogBlockMethodSurrogateClass: CogBlockMethodSurrogate64 nsSendCacheSurrogateClass: NSSendCacheSurrogate64 cPICPrototype: a CogMethodSurrogate64 @5568/16r15C0 cPICEndOfCodeOffset: 209 cPICEndOfCodeLabel: a CogInLineLiteralsX64Compiler (Label 6 @1691) ceMallocTrampoline: nil ceFreeTrampoline: nil ceFFICalloutTrampoline: nil debugBytecodePointers: a Set() debugOpcodeIndices: a Set() disassemblingMethod: nil cogConstituentIndex: nil directedSendUsesBinding: nil processorFrameValid: false codeToDataDelta: 0 varBaseAddress: 2097160 codeZoneIsExecutableNotWritable: false debugAPISelector: nil shortCutTrampolineBlocks: an IdentityDictionary(#ceShortCutTraceBlockActivation:...etc... ioHighResClock: 996 perMethodProfile: nil instructionProfile: nil primitiveGeneratorTable: a CArrayAccessor on: {a CogPrimitiveDescriptor . a CogPrimitiveDescriptor...etc... introspectionDataIndex: nil introspectionData: nil prevBCDescriptor: nil numPushNilsFunction: #v3:Num:Push:Nils: pushNilSizeFunction: #v3PushNilSize:numInitialNils: methodOrBlockNumTemps: nil regArgsHaveBeenPushed: nil simStack: a CArrayAccessor on: {a CogSimStackEntry . a CogSimStackEntry . a CogSimStackEntry...etc... simStackPtr: nil simSpillBase: nil ceCallCogCodePopReceiverArg0Regs: 5248 ceCallCogCodePopReceiverArg1Arg0Regs: 5272 methodAbortTrampolines: a CArrayAccessor on: #(1680 1792 1904 2016) picAbortTrampolines: a CArrayAccessor on: #(2104 2216 2336 2456) picMissTrampolines: a CArrayAccessor on: #(2552 2608 2664 2720) ceCall0ArgsPIC: 5296 ceCall1ArgsPIC: 5328 ceCall2ArgsPIC: 5360 debugStackPointers: nil debugFixupBreaks: a Set() realCECallCogCodePopReceiverArg0Regs: nil realCECallCogCodePopReceiverArg1Arg0Regs: nil deadCode: nil useTwoPaths: nil currentCallCleanUpSize: nil simNativeStack: a CArrayAccessor on: {a CogSimStackNativeEntry . a CogSimStackNativeEntry...etc... simNativeStackPtr: nil simNativeSpillBase: nil simNativeStackSize: nil hasNativeFrame: nil compilationPass: nil StackToRegisterMappingCogit class(Cogit class)>>genAndDis:options: Receiver: StackToRegisterMappingCogit Arguments and temporary variables: methodOrDoitString: (SequenceableCollection>>#indexOf:startingAt:ifAbsent: "a CompiledMethod...etc... optionsDictionaryOrArray: #(#ObjectMemory #Spur64BitCoMemoryManager #ISA #X64) tuple: {a CurrentImageCoInterpreterFacadeFor64BitSpurObjectRepresentation . a StackToRegisterMappingCogit...etc... Receiver's instance variables: superclass: SimpleStackBasedCogit methodDict: a MethodDictionary(size 607) format: 65731 instanceVariables: #('prevBCDescriptor' 'numPushNilsFunction' 'pushNilSizeFunct...etc... organization: ('accessing' simSelf) ('bytecode generator stores' genGenericStorePop:MaybeContextSlotIndex:needsStoreCheck:needsRestoreRcvr:needsImmutabilityCheck:...etc... subclasses: {RegisterAllocatingCogit . SistaCogit . StackToRegisterMappingCogitChecker...etc... name: #StackToRegisterMappingCogit classPool: a Dictionary(#NeedsMergeFixupFlag->nil #NeedsNonMergeFixupFlag->nil ...etc... sharedPools: an OrderedCollection(CogCompilationConstants VMMethodCacheConstants...etc... environment: Smalltalk category: #'VMMaker-JIT' timeStamp: 3873358619 generatorTable: a CArrayAccessor on: {a CogBytecodeDescriptor (genPushReceiverVariableBytecode...etc... primitiveTable: a CArrayAccessor on: {a CogPrimitiveDescriptor . a CogPrimitiveDescriptor...etc... numPushNilsFunction: #v3:Num:Push:Nils: pushNilSizeFunction: #v3PushNilSize:numInitialNils: UndefinedObject>>DoIt Receiver: nil Arguments and temporary variables: Receiver's instance variables: nil Compiler>>evaluateCue:ifFail: Receiver: a Compiler Arguments and temporary variables: aCue: a CompilationCue failBlock: [closure] in Compiler>>evaluateCue:ifFail:logged: methodNode: DoIt ^ StackToRegisterMappingCogit genAndDis: SequenceableCollecti...etc... method: (UndefinedObject>>#DoIt "a CompiledMethod(4121528)") value: nil Receiver's instance variables: parser: a Parser cue: a CompilationCue Compiler>>evaluateCue:ifFail:logged: Receiver: a Compiler Arguments and temporary variables: aCue: a CompilationCue failBlock: [closure] in [] in SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo:...etc... logFlag: true value: nil Receiver's instance variables: parser: a Parser cue: a CompilationCue Compiler>>evaluate:in:to:environment:notifying:ifFail:logged: Receiver: a Compiler Arguments and temporary variables: textOrStream: a ReadStream aContext: nil receiver: nil anEnvironment: Smalltalk aRequestor: a SmalltalkEditor failBlock: [closure] in [] in SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo:...etc... logFlag: true Receiver's instance variables: parser: a Parser cue: a CompilationCue [] in SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo: Receiver: a SmalltalkEditor Arguments and temporary variables: aBlock: [closure] in SmalltalkEditor(TextEditor)>>evaluateSelection result: nil rcvr: nil ctxt: nil Receiver's instance variables: morph: a TextMorphForEditView(216877) model: a Workspace paragraph: a NewParagraph markBlock: a CharacterBlock with index 613 and character Character return and r...etc... pointBlock: a CharacterBlock with index 767 and rectangle 422@290 corner: 422@3...etc... beginTypeInIndex: nil emphasisHere: {a TextColor code: Color black} lastParenLocation: nil otherInterval: (613 to: 766) oldInterval: (627 to: 626) typeAhead: a WriteStream history: a TextEditorCommandHistory FullBlockClosure(BlockClosure)>>on:do: Receiver: [closure] in SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo: Arguments and temporary variables: exceptionOrExceptionSet: OutOfScopeNotification handlerAction: [closure] in SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo:...etc... handlerActive: true handlerRearmed: false Receiver's instance variables: outerContext: SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo: startpcOrMethod: ([] in TextEditor>>#evaluateSelectionAndDo: "a CompiledBlock(3...etc... numArgs: 0 receiver: a SmalltalkEditor SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo: Receiver: a SmalltalkEditor Arguments and temporary variables: aBlock: [closure] in SmalltalkEditor(TextEditor)>>evaluateSelection result: nil rcvr: nil ctxt: nil Receiver's instance variables: morph: a TextMorphForEditView(216877) model: a Workspace paragraph: a NewParagraph markBlock: a CharacterBlock with index 613 and character Character return and r...etc... pointBlock: a CharacterBlock with index 767 and rectangle 422@290 corner: 422@3...etc... beginTypeInIndex: nil emphasisHere: {a TextColor code: Color black} lastParenLocation: nil otherInterval: (613 to: 766) oldInterval: (627 to: 626) typeAhead: a WriteStream history: a TextEditorCommandHistory SmalltalkEditor(TextEditor)>>evaluateSelection Receiver: a SmalltalkEditor Arguments and temporary variables: Receiver's instance variables: morph: a TextMorphForEditView(216877) model: a Workspace paragraph: a NewParagraph markBlock: a CharacterBlock with index 613 and character Character return and r...etc... pointBlock: a CharacterBlock with index 767 and rectangle 422@290 corner: 422@3...etc... beginTypeInIndex: nil emphasisHere: {a TextColor code: Color black} lastParenLocation: nil otherInterval: (613 to: 766) oldInterval: (627 to: 626) typeAhead: a WriteStream history: a TextEditorCommandHistory SmalltalkEditor(TextEditor)>>doIt Receiver: a SmalltalkEditor Arguments and temporary variables: Receiver's instance variables: morph: a TextMorphForEditView(216877) model: a Workspace paragraph: a NewParagraph markBlock: a CharacterBlock with index 613 and character Character return and r...etc... pointBlock: a CharacterBlock with index 767 and rectangle 422@290 corner: 422@3...etc... beginTypeInIndex: nil emphasisHere: {a TextColor code: Color black} lastParenLocation: nil otherInterval: (613 to: 766) oldInterval: (627 to: 626) typeAhead: a WriteStream history: a TextEditorCommandHistory SmalltalkEditor(TextEditor)>>doIt: Receiver: a SmalltalkEditor Arguments and temporary variables: aKeyboardEvent: [104@339 keystroke '<Ctrl-Cmd-d>' (100) 3090] Receiver's instance variables: morph: a TextMorphForEditView(216877) model: a Workspace paragraph: a NewParagraph markBlock: a CharacterBlock with index 613 and character Character return and r...etc... pointBlock: a CharacterBlock with index 767 and rectangle 422@290 corner: 422@3...etc... beginTypeInIndex: nil emphasisHere: {a TextColor code: Color black} lastParenLocation: nil otherInterval: (613 to: 766) oldInterval: (627 to: 626) typeAhead: a WriteStream history: a TextEditorCommandHistory SmalltalkEditor(TextEditor)>>dispatchOnKeyboardEvent: Receiver: a SmalltalkEditor Arguments and temporary variables: aKeyboardEvent: [104@339 keystroke '<Ctrl-Cmd-d>' (100) 3090] honorCommandKeys: true typedChar: $d Receiver's instance variables: morph: a TextMorphForEditView(216877) model: a Workspace paragraph: a NewParagraph markBlock: a CharacterBlock with index 613 and character Character return and r...etc... pointBlock: a CharacterBlock with index 767 and rectangle 422@290 corner: 422@3...etc... beginTypeInIndex: nil emphasisHere: {a TextColor code: Color black} lastParenLocation: nil otherInterval: (613 to: 766) oldInterval: (627 to: 626) typeAhead: a WriteStream history: a TextEditorCommandHistory SmalltalkEditor(TextEditor)>>keyStroke: Receiver: a SmalltalkEditor Arguments and temporary variables: anEvent: [104@339 keystroke '<Ctrl-Cmd-d>' (100) 3090] Receiver's instance variables: morph: a TextMorphForEditView(216877) model: a Workspace paragraph: a NewParagraph markBlock: a CharacterBlock with index 613 and character Character return and r...etc... pointBlock: a CharacterBlock with index 767 and rectangle 422@290 corner: 422@3...etc... beginTypeInIndex: nil emphasisHere: {a TextColor code: Color black} lastParenLocation: nil otherInterval: (613 to: 766) oldInterval: (627 to: 626) typeAhead: a WriteStream history: a TextEditorCommandHistory [] in [] in TextMorphForEditView(TextMorph)>>keyStroke: Receiver: a TextMorphForEditView(216877) Arguments and temporary variables: evt: [104@339 keystroke '<Ctrl-Cmd-d>' (100) 3090] action: nil Receiver's instance variables: bounds: 0@0 corner: 753@308 owner: a TransformMorph(217319) submorphs: #() fullBounds: 0@0 corner: 753@308 color: Color black extension: a MorphExtension (1612197) [other: (unfocusedSelectionColor -> (Col...etc... borderWidth: 0 borderColor: Color black textStyle: a TextStyle(6) Bitmap DejaVu Sans 10.5pt 96ppi 14px Normal text: a Text for '"See what code the JIT generates for as method in the current...etc... wrapFlag: true paragraph: a NewParagraph editor: a SmalltalkEditor container: nil predecessor: nil successor: nil backgroundColor: nil margins: 3@2 corner: 3@2 readOnly: false autoFit: true plainTextOnly: false numCharactersPerLine: nil editView: a PluggableTextMorphPlus(1615011) acceptOnCR: false TextMorphForEditView(TextMorph)>>handleInteraction:fromEvent: Receiver: a TextMorphForEditView(216877) Arguments and temporary variables: interactionBlock: [closure] in [] in TextMorphForEditView(TextMorph)>>keyStroke:...etc... evt: [104@339 keystroke '<Ctrl-Cmd-d>' (100) 3090] oldEditor: a SmalltalkEditor oldParagraph: a NewParagraph oldText: a Text for '"See what code the JIT generates for as method in the curr...etc... oldSelection: an OrderedCollection(3@242 corner: 750@290 3@290 corner: 422@306)...etc... Receiver's instance variables: bounds: 0@0 corner: 753@308 owner: a TransformMorph(217319) submorphs: #() fullBounds: 0@0 corner: 753@308 color: Color black extension: a MorphExtension (1612197) [other: (unfocusedSelectionColor -> (Col...etc... borderWidth: 0 borderColor: Color black textStyle: a TextStyle(6) Bitmap DejaVu Sans 10.5pt 96ppi 14px Normal text: a Text for '"See what code the JIT generates for as method in the current...etc... wrapFlag: true paragraph: a NewParagraph editor: a SmalltalkEditor container: nil predecessor: nil successor: nil backgroundColor: nil margins: 3@2 corner: 3@2 readOnly: false autoFit: true plainTextOnly: false numCharactersPerLine: nil editView: a PluggableTextMorphPlus(1615011) acceptOnCR: false TextMorphForEditView>>handleInteraction:fromEvent: Receiver: a TextMorphForEditView(216877) Arguments and temporary variables: interActionBlock: [closure] in [] in TextMorphForEditView(TextMorph)>>keyStroke:...etc... evt: [104@339 keystroke '<Ctrl-Cmd-d>' (100) 3090] Receiver's instance variables: bounds: 0@0 corner: 753@308 owner: a TransformMorph(217319) submorphs: #() fullBounds: 0@0 corner: 753@308 color: Color black extension: a MorphExtension (1612197) [other: (unfocusedSelectionColor -> (Col...etc... borderWidth: 0 borderColor: Color black textStyle: a TextStyle(6) Bitmap DejaVu Sans 10.5pt 96ppi 14px Normal text: a Text for '"See what code the JIT generates for as method in the current...etc... wrapFlag: true paragraph: a NewParagraph editor: a SmalltalkEditor container: nil predecessor: nil successor: nil backgroundColor: nil margins: 3@2 corner: 3@2 readOnly: false autoFit: true plainTextOnly: false numCharactersPerLine: nil editView: a PluggableTextMorphPlus(1615011) acceptOnCR: false --- The full stack --- UndefinedObject(Object)>>doesNotUnderstand: #isCMBlock StackToRegisterMappingCogit(Cogit)>>disassembleMethod:on: StackToRegisterMappingCogit(Cogit)>>disassembleMethod: StackToRegisterMappingCogit class(Cogit class)>>genAndDis:options: UndefinedObject>>DoIt Compiler>>evaluateCue:ifFail: Compiler>>evaluateCue:ifFail:logged: Compiler>>evaluate:in:to:environment:notifying:ifFail:logged: [] in SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo: FullBlockClosure(BlockClosure)>>on:do: SmalltalkEditor(TextEditor)>>evaluateSelectionAndDo: SmalltalkEditor(TextEditor)>>evaluateSelection SmalltalkEditor(TextEditor)>>doIt SmalltalkEditor(TextEditor)>>doIt: SmalltalkEditor(TextEditor)>>dispatchOnKeyboardEvent: SmalltalkEditor(TextEditor)>>keyStroke: [] in [] in TextMorphForEditView(TextMorph)>>keyStroke: TextMorphForEditView(TextMorph)>>handleInteraction:fromEvent: TextMorphForEditView>>handleInteraction:fromEvent: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - [] in TextMorphForEditView(TextMorph)>>keyStroke: StandardToolSet class>>codeCompletionAround:textMorph:keyStroke: ToolSet class>>codeCompletionAround:textMorph:keyStroke: TextMorphForEditView(TextMorph)>>keyStroke: TextMorphForEditView>>keyStroke: TextMorphForEditView(Morph)>>handleKeystroke: TextMorphForEditView(TextMorph)>>handleKeystroke: KeyboardEvent>>sentTo: TextMorphForEditView(Morph)>>handleEvent: TextMorphForEditView(Morph)>>handleFocusEvent: MorphicEventDispatcher>>doHandlingForFocusEvent:with: MorphicEventDispatcher>>dispatchFocusEvent:with: TextMorphForEditView(Morph)>>processFocusEvent:using: TextMorphForEditView(Morph)>>processFocusEvent: [] in [] in [] in HandMorph>>sendFocusEvent:to:clear: [] in ActiveEventVariable class(DynamicVariable class)>>value:during: FullBlockClosure(BlockClosure)>>ensure: ActiveEventVariable class(DynamicVariable class)>>value:during: [] in ActiveEventVariable class>>value:during: FullBlockClosure(BlockClosure)>>ensure: ActiveEventVariable class>>value:during: KeyboardEvent(MorphicEvent)>>becomeActiveDuring: [] in [] in HandMorph>>sendFocusEvent:to:clear: [] in ActiveHandVariable class(DynamicVariable class)>>value:during: FullBlockClosure(BlockClosure)>>ensure: ActiveHandVariable class(DynamicVariable class)>>value:during: [] in ActiveHandVariable class>>value:during: FullBlockClosure(BlockClosure)>>ensure: ActiveHandVariable class>>value:during: HandMorph>>becomeActiveDuring: [] in HandMorph>>sendFocusEvent:to:clear: [] in ActiveWorldVariable class(DynamicVariable class)>>value:during: FullBlockClosure(BlockClosure)>>ensure: ActiveWorldVariable class(DynamicVariable class)>>value:during: [] in ActiveWorldVariable class>>value:during: FullBlockClosure(BlockClosure)>>ensure: ActiveWorldVariable class>>value:during: PasteUpMorph>>becomeActiveDuring: HandMorph>>sendFocusEvent:to:clear: HandMorph>>sendEvent:focus:clear: HandMorph>>sendKeyboardEvent: HandMorph>>handleEvent: -- and more not shown --

2 3

VM Maker: VMMaker.threaded-LM.3341.mcz
by commits＠source.squeak.org 03 Oct '23

03 Oct '23

1 0

VM Maker: VMMaker.threaded-LM.3339.mcz
by commits＠source.squeak.org 03 Oct '23

03 Oct '23

Leon Matthes uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.threaded-LM.3339.mcz ==================== Summary ==================== Name: VMMaker.threaded-LM.3339 Author: LM Time: 28 September 2023, 4:12:53.651627 pm UUID: 0d2d7578-99c6-415e-a3e4-bb64bc493a2a Ancestors: VMMaker.oscog-eem.3338 Make vmThread>>#state an atomic_int For this to work in both Smalltalk and the generated C code, the #state and #state: accessors can no longer be used! Instead, the vmThreadState and setVmThreadState functions should be called. Otherwise the generation doesn't work out, as Slang can't handle the atomic_load and atomic_store in the accessors. =============== Diff against VMMaker.oscog-eem.3338 =============== Item was changed: ----- Method: CoInterpreterMT>>assertSaneThreadAndProcess (in category 'debug support') ----- assertSaneThreadAndProcess <inline: true> self assert: cogThreadManager vmIsOwned. + self assert: cogThreadManager currentVMThread vmThreadState = CTMAssignableOrInVM. - self assert: cogThreadManager currentVMThread state = CTMAssignableOrInVM. self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. cogThreadManager assertValidProcessorStackPointersForIndex: cogThreadManager getVMOwner! Item was changed: ----- Method: CoInterpreterMT>>disownVM: (in category 'vm scheduling') ----- disownVM: flags "Release the VM to other threads and answer the current thread's index. Currently valid flags: DisownVMForFFICall - informs the VM that it is entering an FFI call DisownVMForThreading - informs the VM that it is entering code during which threading should be permitted OwnVMForeignThreadFlag - indicates lowest-level entry from a foreign thread - not to be used explicitly by clients - only set by ownVMFromUnidentifiedThread VMAlreadyOwnedHenceDoNotDisown - indicates an ownVM from a callback was made when the vm was still owned. - not to be used explicitly by clients - only set by ownVMFromUnidentifiedThread This is the entry-point for plugins and primitives that wish to release the VM while performing some operation that may potentially block, and for callbacks returning back to some blocking operation. If this thread does not reclaim the VM before- hand then when the next heartbeat occurs the thread manager will schedule a thread to acquire the VM which may start running the VM in place of this thread. N.B. Most of the state needed to resume after preemption is set in preemptDisowningThread." <public> <inline: false> | vmThread result | self assert: (flags >= 0 and: [flags < (1 bitShift: DisownFlagsShift)]). self assert: self successful. cogit recordEventTrace ifTrue: [self recordTrace: TraceDisownVM thing: (objectMemory integerObjectOf: flags) source: 0]. processHasThreadAffinity ifFalse: [willNotThreadWarnCount < 10 ifTrue: [self print: 'warning: VM parameter 48 indicates Process doesn''t have threadId; VM will not thread'; cr. willNotThreadWarnCount := willNotThreadWarnCount + 1]]. vmThread := cogThreadManager currentVMThread. (flags anyMask: VMAlreadyOwnedHenceDoNotDisown) ifTrue: [disowningVMThread := vmThread. + vmThread setVmThreadState: CTMUnavailable. - vmThread state: CTMUnavailable. ^0]. self assertCStackPointersBelongToCurrentThread. self assertValidNewMethodPropertyFlags. self cCode: '' inSmalltalk: [cogThreadManager saveRegisterStateForCurrentProcess. cogThreadManager clearRegisterStates.]. (flags anyMask: DisownVMForProcessorRelinquish) ifTrue: [| proc | (proc := objectMemory splObj: foreignCallbackProcessSlot) ~= objectMemory nilObject ifTrue: [foreignCallbackPriority := self quickFetchInteger: PriorityIndex ofObject: proc]. relinquishing := true. self sqLowLevelMFence]. disownCount := disownCount + 1. "If we're disowning the VM because there's no active process to run, there's nothing to preempt later, so don't indicate that there's a disowningVMThread that needs to be restored later." self activeProcess ~= objectMemory nilObject ifTrue: [disowningVMThread := vmThread]. "OwnVMForeignThreadFlag indicates lowest-level of entry by a foreign thread. If that's where we are then release the vmThread. Otherwise indicate the vmThread is off doing something outside of the VM." (flags anyMask: OwnVMForeignThreadFlag) ifTrue: ["I don't think this is quite right. Josh's use case is creating some foreign thread and then registering it with the VM. That's not the same as binding a process to a foreign thread given that the foreign callback process is about to terminate anyway (it is returning from a callback here). So do we need an additional concept, that of a vmThread being either of the set known to the VM or floating?" self flag: 'issue with registering foreign threads with the VM'. (self isBoundProcess: self activeProcess) ifFalse: [cogThreadManager unregisterVMThread: vmThread]] + ifFalse: [vmThread setVmThreadState: CTMUnavailable]. - ifFalse: [vmThread state: CTMUnavailable]. result := ((vmThread index bitShift: DisownFlagsShift) bitOr: (activeProcessAffined ifTrue: [0] ifFalse: [ProcessUnaffinedOnDisown])) bitOr: flags. cogThreadManager releaseVM. ^result! Item was changed: ----- Method: CoInterpreterMT>>mapInterpreterOops (in category 'object memory support') ----- mapInterpreterOops "Map all oops in the interpreter's state to their new values during garbage collection or a become: operation." "Assume: All traced variables contain valid oops." <var: #vmThread type: #'CogVMThread *'> super mapInterpreterOops. "Per-thread state; trace each thread's own newMethod and stack of awol processes." 1 to: cogThreadManager getNumThreads do: [:i| | vmThread | vmThread := cogThreadManager vmThreadAt: i. + vmThread vmThreadState ~= CTMUninitialized ifTrue: - vmThread state ifNotNil: [(vmThread newMethodOrNull notNil and: [objectMemory shouldRemapOop: vmThread newMethodOrNull]) ifTrue: [vmThread newMethodOrNull: (objectMemory remapObj: vmThread newMethodOrNull)]. 0 to: vmThread awolProcIndex - 1 do: [:j| (objectMemory shouldRemapOop: (vmThread awolProcesses at: j)) ifTrue: [vmThread awolProcesses at: j put: (objectMemory remapObj: (vmThread awolProcesses at: j))]]]]! Item was changed: ----- Method: CoInterpreterMT>>markAndTraceInterpreterOops: (in category 'object memory support') ----- markAndTraceInterpreterOops: fullGCFlag "Override to mark the awolProcesses" <var: #vmThread type: #'CogVMThread *'> super markAndTraceInterpreterOops: fullGCFlag. "Per-thread state; trace each thread's own newMethod and stack of awol processes." 1 to: cogThreadManager getNumThreads do: [:i| | vmThread | vmThread := cogThreadManager vmThreadAt: i. + vmThread vmThreadState ~= CTMUninitialized ifTrue: - vmThread state ifNotNil: [vmThread newMethodOrNull ifNotNil: [objectMemory markAndTrace: vmThread newMethodOrNull]. 0 to: vmThread awolProcIndex - 1 do: [:j| objectMemory markAndTrace: (vmThread awolProcesses at: j)]]]! Item was changed: ----- Method: CoInterpreterMT>>ownVMFromUnidentifiedThread (in category 'vm scheduling') ----- ownVMFromUnidentifiedThread "Attempt to take ownership from a thread that as yet doesn't know its index. This supports callbacks where the callback could originate from any thread. Answer 0 if the owning thread is known to the VM. Answer 1 if the owning thread is unknown to the VM and now owns the VM. Answer -1 if the owning thread is unknown to the VM and fails to own the VM. Answer -2 if the owning thread is unknown to the VM and there is no foreign callback process installed." | count threadIndex vmThread | <var: #vmThread type: #'CogVMThread *'> <inline: false> self cCode: [] inSmalltalk: [self halt: 'TODO: Implement processor register switching']. (threadIndex := cogThreadManager ioGetThreadLocalThreadIndex) ~= 0 ifTrue: [ "this is a callback from a known thread" (cogThreadManager vmOwnerIs: threadIndex) ifTrue: "the VM has not been disowned" [self assert: (disowningVMThread isNil or: [disowningVMThread = self currentVMThread]). disowningVMThread := nil. + self currentVMThread setVmThreadState: CTMAssignableOrInVM. - self currentVMThread state: CTMAssignableOrInVM. ^VMAlreadyOwnedHenceDoNotDisown]. ^self ownVM: threadIndex]. foreignCallbackPriority = 0 ifTrue: [^-2]. count := 0. "If the current thread doesn't have an index it's new to the vm and we need to allocate a new threadInfo, failing if we can't. We also need a process in the foreignCallbackProcessSlot upon which to run the thread's eventual callback." [[cogThreadManager tryLockVMOwnerTo: cogThreadManager ioCurrentOSThread asUnsignedInteger] whileFalse: [self waitingPriorityIsAtLeast: foreignCallbackPriority. cogThreadManager ioTransferTimeslice]. (objectMemory splObj: foreignCallbackProcessSlot) ~= objectMemory nilObject] whileFalse: [cogThreadManager releaseVM. (count := count + 1) > 1000 ifTrue: [^-2]. cogThreadManager ioMilliSleep: 1]. vmThread := cogThreadManager unusedThreadInfo. "N.B. Keep the VM locked anonymously so that we reserve the non-nil ForeignCallbackProcess for this thread, avoiding the race between competing foreign callbacks. The acquireVMFor: in ownVM: will set the vmOwner to the actual index. So only unlock on failure." vmThread ifNil: [cogThreadManager releaseVM. ^-1]. cogThreadManager setVMOwner: vmThread index. vmThread + setVmThreadState: CTMWantingOwnership; - state: CTMWantingOwnership; priority: foreignCallbackPriority. cogThreadManager registerVMThread: vmThread. ^self ownVM: vmThread index + OwnVMForeignThreadFlag! Item was changed: ----- Method: CoInterpreterMT>>preemptDisowningThread (in category 'vm scheduling') ----- preemptDisowningThread "Set the relevant state for disowningVMThread so that it can resume after being preempted and set disowningVMThread to nil to indicate preemption. N.B. This should only be sent from checkPreemptionOfDisowningThread. There are essentially four things to do. a) save the VM's notion of the current C stack pointers; these are pointers into a thread's stack and must be saved and restored in thread switch. b) save the VM's notion of the current Smalltalk execution point. This is simply the suspend half of a process switch that saves the current context in the current process. c) add the process to the thread's set of AWOL processes so that the scheduler won't try to run the process while the thread has disowned the VM. d) save the in-primitive VM state, newMethod and argumentCount ownVM: will restore the VM context as of disownVM: from the above when it finds it has been preempted." | activeProc activeContext preemptedThread | <var: #preemptedThread type: #'CogVMThread *'> <inline: false> self assert: disowningVMThread notNil. + self assert: (disowningVMThread vmThreadState = CTMUnavailable + or: [disowningVMThread vmThreadState = CTMWantingOwnership]). - self assert: (disowningVMThread state = CTMUnavailable - or: [disowningVMThread state = CTMWantingOwnership]). self assertCStackPointersBelongToDisowningThread. cogit recordEventTrace ifTrue: [self recordTrace: TracePreemptDisowningThread thing: (objectMemory integerObjectOf: disowningVMThread index) source: 0]. disowningVMThread cStackPointer: CStackPointer. disowningVMThread cFramePointer: CFramePointer. activeProc := self activeProcess. self assert: (objectMemory fetchPointer: MyListIndex ofObject: activeProc) = objectMemory nilObject. objectMemory storePointer: MyListIndex ofObject: activeProc withValue: (objectMemory splObj: ProcessInExternalCodeTag). activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer. objectMemory storePointer: SuspendedContextIndex ofObject: activeProc withValue: activeContext. "The instructionPointer must be pushed because the convention for inactive stack pages is that the instructionPointer is top of stack. We need to know if this primitive is called from machine code because the invariant that the return pc of an interpreter callee calling a machine code caller is ceReturnToInterpreterPC must be maintained." self push: instructionPointer. self externalWriteBackHeadFramePointers. "Since pushing the awol process may realloc disowningVMThread we need to reassign. But since we're going to nil disowningVMThread anyway we can assign to a local." preemptedThread := cogThreadManager pushAWOLProcess: activeProc on: disowningVMThread. disowningVMThread := nil. preemptedThread priority: (self quickFetchInteger: PriorityIndex ofObject: activeProc). (self ownerIndexOfProcess: activeProc) = 0 ifTrue: [self setOwnerIndexOfProcess: activeProc to: preemptedThread index bind: false]. preemptedThread newMethodOrNull: newMethod; argumentCount: argumentCount; inMachineCode: instructionPointer <= objectMemory startOfMemory! Item was changed: ----- Method: CoInterpreterMT>>primitiveRelinquishProcessor (in category 'I/O primitives') ----- primitiveRelinquishProcessor "Relinquish the processor for up to the given number of microseconds. The exact behavior of this primitive is platform dependent. Override to check for waiting threads." | microSecs threadIndexAndFlags currentCStackPointer currentCFramePointer | <var: #currentCStackPointer type: #'volatile usqIntptr_t'> <var: #currentCFramePointer type: #'volatile usqIntptr_t'> microSecs := self stackTop. (objectMemory isIntegerObject: microSecs) ifFalse: [^self primitiveFail]. self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. self assert: relinquishing not. "DO NOT allow relinquishing the processor while we are profiling since this may skew the time base for our measures (it may reduce processor speed etc). Instead we go full speed, therefore measuring the precise time we spend in the inner idle loop as a busy loop." nextProfileTick = 0 ifTrue: "Presumably we have nothing to do; this primitive is typically called from the background process. So we should /not/ try and activate any threads in the pool; they will waste cycles finding there is no runnable process, and will cause a VM abort if no runnable process is found. But we /do/ want to allow FFI calls that have completed, or callbacks a chance to get into the VM; they do have something to do. DisownVMForProcessorRelinquish indicates this." [currentCStackPointer := CStackPointer. currentCFramePointer := CFramePointer. threadIndexAndFlags := self disownVM: DisownVMForProcessorRelinquish. self assert: relinquishing. self ioRelinquishProcessorForMicroseconds: (objectMemory integerValueOf: microSecs). self assert: relinquishing. self ownVM: threadIndexAndFlags. self assert: relinquishing not. + self assert: cogThreadManager currentVMThread vmThreadState = CTMAssignableOrInVM. - self assert: cogThreadManager currentVMThread state = CTMAssignableOrInVM. self assert: currentCStackPointer = CStackPointer. self assert: currentCFramePointer = CFramePointer. "In simulation we allow ioRelinquishProcessorForMicroseconds: to fail so that we can arrange that the simulator responds to input events promptly. This *DOES NOT HAPPEN* in the real vm." self cCode: [] inSmalltalk: [primFailCode ~= 0 ifTrue: [^self]]]. self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. self pop: 1 "microSecs; leave rcvr on stack"! Item was changed: ----- Method: CoInterpreterMT>>printAllStacks (in category 'debug printing') ----- printAllStacks "Print all the stacks of all running processes, including those that are currently suspended. Override to print the AWOL processes." super printAllStacks. self cr; print: 'awol processes'. 1 to: cogThreadManager getNumThreads do: [:i| | vmThread | vmThread := cogThreadManager vmThreadAt: i. + vmThread vmThreadState ~= CTMUninitialized ifTrue: - vmThread state ifNotNil: [vmThread awolProcIndex > 0 ifTrue: [self cr; print: 'thread '; printNum: i. 0 to: vmThread awolProcIndex - 1 do: [:j| self printProcessStack: (vmThread awolProcesses at: j)]]]]! Item was changed: ----- Method: CoInterpreterMT>>threadSchedulingLoopImplementation: (in category 'vm scheduling') ----- threadSchedulingLoopImplementation: vmThread "Enter a loop attempting to run the VM with the highest priority process and blocking on the thread's OS semaphore when unable to run that process. We will return to this via threadSwitchIfNecessary:from: which is called in the middle of transferTo:from: once the active process has been stored in the scheduler." <var: #vmThread type: #'CogVMThread *'> <inline: false> self _setjmp: vmThread reenterThreadSchedulingLoop. + [self assert: vmThread vmThreadState = CTMAssignableOrInVM. - [self assert: vmThread state = CTMAssignableOrInVM. (cogThreadManager tryLockVMOwnerTo: vmThread index) ifTrue: ["Yay, we're the VM owner!!" "If relinquishing is true, then primitiveRelinquishProcessor has disowned the VM and only a returning call or callback should take ownership in that case." relinquishing ifFalse: [self tryToExecuteSmalltalk: vmThread]. self disownVM: DisownVMForThreading.]. cogThreadManager waitForWork: vmThread. true] whileTrue! Item was changed: ----- Method: CoInterpreterMT>>threadSwitchIfNecessary:from: (in category 'process primitive support') ----- threadSwitchIfNecessary: newProc from: sourceCode "Invoked from transferTo:from: or primitiveProcessBindToThreadId to switch threads if the new process is bound or affined to some other thread." | newProcOwnerIndex vmThread activeContext | self assert: (cogThreadManager vmOwnerIs: cogThreadManager ioGetThreadLocalThreadIndex). deferThreadSwitch ifTrue: [^self]. cogThreadManager assertValidProcessorStackPointersForIndex: cogThreadManager getVMOwner. "If the current process is unaffined or it is affined to the current thread we're ok to run, but we should yield asap if a higher-priority thread wants the VM." newProcOwnerIndex := self ownerIndexOfProcess: newProc. ((activeProcessAffined := newProcOwnerIndex ~= 0) and: [(cogThreadManager vmOwnerIsCompatibleWith: newProcOwnerIndex) not]) ifFalse: [(self quickFetchInteger: PriorityIndex ofObject: newProc) < maxWaitingPriority ifTrue: [checkThreadActivation := true. self forceInterruptCheck]. ^self]. "The current process is affined to a thread, but not to the current owner. So switch to that owner." self cCode: '' inSmalltalk: [transcript ensureCr; f: 'threadSwitchIfNecessary: %08x from: %s(%d) owner %d -> %d\n' printf: { newProc. TraceSources at: sourceCode. sourceCode. cogThreadManager getVMOwner. newProcOwnerIndex }]. "We at least need to externalize the stack pointers to enable a thread switch..." (objectMemory fetchPointer: SuspendedContextIndex ofObject: newProc) = objectMemory nilObject ifTrue: [self assert: newProc = self activeProcess. self push: instructionPointer. self externalWriteBackHeadFramePointers. false ifTrue: "If the activeProcess doesn't have a context yet, it needs one from which the new thread can resume execution." [activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer. objectMemory storePointer: SuspendedContextIndex ofObject: newProc withValue: activeContext]]. newProcOwnerIndex < 0 ifTrue: [self assert: newProcOwnerIndex negated = cogThreadManager getVMOwner. vmThread := cogThreadManager ensureWillingThread. self deny: vmThread index = cogThreadManager getVMOwner. self assert: (cogThreadManager threadIndex: vmThread index isCompatibleWith: newProcOwnerIndex)] ifFalse: [vmThread := cogThreadManager vmThreadAt: newProcOwnerIndex. vmThread priority: (self quickFetchInteger: PriorityIndex ofObject: newProc). + vmThread vmThreadState = CTMUnavailable ifTrue: + [vmThread setVmThreadState: CTMWantingOwnership]]. - vmThread state = CTMUnavailable ifTrue: - [vmThread state: CTMWantingOwnership]]. self returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source: CSSwitchIfNeccessary! Item was changed: ----- Method: CoInterpreterMT>>transferTo:from: (in category 'process primitive support') ----- transferTo: newProc from: sourceCode "Record a process to be awoken on the next interpreter cycle. Override to potentially switch threads either if the new process is bound to another thread, or if there is no runnable process but there is a waiting thread. Note that the abort on no runnable process has beeen moved here from wakeHighestPriority." | sched oldProc activeContext | <inline: false> statProcessSwitch := statProcessSwitch + 1. self push: instructionPointer. self externalWriteBackHeadFramePointers. self assertValidExecutionPointe: instructionPointer r: framePointer s: stackPointer. "ensureMethodIsCogged: in makeBaseFrameFor: in externalSetStackPageAndPointersForSuspendedContextOfProcess: below may do a code compaction. Nil instructionPointer to avoid it getting pushed twice." instructionPointer := 0. sched := self schedulerPointer. oldProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched. self recordContextSwitchFrom: oldProc in: sourceCode. activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer + objectMemory wordSize. objectMemory storePointer: SuspendedContextIndex ofObject: oldProc withValue: activeContext. newProc ifNil: ["Two possibilities. One, there is at least one thread waiting to own the VM in which case it should be activated. Two, there are no processes to run and so abort." cogThreadManager willingVMThread ifNotNil: [:vmThread| + vmThread vmThreadState = CTMWantingOwnership ifTrue: - vmThread state = CTMWantingOwnership ifTrue: [self returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source: sourceCode]]. self error: 'scheduler could not find a runnable process']. "Switch to the new process" objectMemory storePointer: ActiveProcessIndex ofObject: sched withValue: newProc; storePointerUnchecked: MyListIndex ofObject: newProc withValue: objectMemory nilObject. self externalSetStackPageAndPointersForSuspendedContextOfProcess: newProc. "Finally thread switch if required" self threadSwitchIfNecessary: newProc from: sourceCode! Item was changed: ----- Method: CogThreadManager class>>initialize (in category 'class initialization') ----- initialize "CogThreadManager initialize" + CTMUninitialized := 0. CTMInitializing := 1. CTMUnavailable := 2. "off doing its own thing and not available to run the VM, e.g. calling-out." CTMAssignableOrInVM := 3. "either owning the VM or blocked on its osSemaphore available for VM work" CTMWantingOwnership := 4. "with something specific to do in the VM (e.g. return a result from a call-out, make a call-back)" "Define the size of the stack of processes at time of disown/own." AWOLProcessesIncrement := 4! Item was changed: ----- Method: CogThreadManager>>acquireVMFor: (in category 'public api') ----- acquireVMFor: threadIndex "Attempt to acquire the VM, eventually blocking until it becomes available. Spin until the maxWaitingPriority has been updated if it is lower than this thread's priority." <returnTypeC: #'CogVMThread *'> | vmThread | <var: #vmThread type: #'CogVMThread *'> self assert: threadIndex = self ioGetThreadLocalThreadIndex. vmThread := self vmThreadAt: threadIndex. + self assert: (vmThread vmThreadState = CTMUnavailable + or: [vmThread vmThreadState = CTMWantingOwnership]). - self assert: (vmThread state = CTMUnavailable - or: [vmThread state = CTMWantingOwnership]). (self tryLockVMOwnerTo: threadIndex) + ifTrue: [vmThread setVmThreadState: CTMAssignableOrInVM] - ifTrue: [vmThread state: CTMAssignableOrInVM] ifFalse: + [vmThread setVmThreadState: CTMWantingOwnership. - [vmThread state: CTMWantingOwnership. [(self vmOwnerIs: threadIndex) or: [self tryLockVMOwnerTo: threadIndex]] whileFalse: [vmThread priority ifNotNil: [coInterpreter waitingPriorityIsAtLeast: vmThread priority]. (self vmOwnerIs: threadIndex) ifFalse: [self ioWaitOnOSSemaphore: (self addressOf: vmThread osSemaphore)]]]. coInterpreter assertProcessorStackPointersBelongToCurrentThread. vmOSThread := vmThread osThread. + vmThread setVmThreadState: CTMAssignableOrInVM. - vmThread state: CTMAssignableOrInVM. ^vmThread! Item was changed: ----- Method: CogThreadManager>>ensureRunningVMThread: (in category 'public api') ----- ensureRunningVMThread: vmIsRelinquishing "Called from checkVMOwnershipFromHeartbeat if the VM is unowned. Hence we are in the heartbeat thread. The race is against that thread owning the VM and against foreign callbacks." <returnTypeC: #void> <var: #vmThread type: #'CogVMThread *'> self willingVMThread ifNotNil: + [:vmThread| | threadState | + threadState := vmThread vmThreadState. - [:vmThread| "If the VM is relinquishing the processor then only schedule a thread if it has work to do." (vmIsRelinquishing + and: [threadState ~= CTMWantingOwnership]) ifTrue: - and: [vmThread state ~= CTMWantingOwnership]) ifTrue: [^self]. + self assert: ((threadState = CTMAssignableOrInVM + or: [threadState = CTMInitializing]) + or: [threadState = CTMWantingOwnership]). - self assert: ((vmThread state = CTMAssignableOrInVM - or: [vmThread state = CTMInitializing]) - or: [vmThread state = CTMWantingOwnership]). (self tryLockVMOwnerTo: vmThread index) ifFalse: "someone beat us to it..." [^self]. vmOSThread := vmThread osThread. "release the thread from its blocking loop" self ioSignalOSSemaphore: (self addressOf: vmThread osSemaphore). self ioTransferTimeslice. "self cCode: [coInterpreter print: 'ERVT signalled '; printNum: vmThread index; cr]." ^self]. "If the VM is relinquishing the processor then only schedule a thread if it has work to do (willingVMThread not nil above). If we have failed to allocate thread storage before there is no point continuing to try to do so. By this time we should have quite a few threads in the pool." (vmIsRelinquishing or: [memoryIsScarce]) ifTrue: [^self]. self unusedThreadInfo ifNotNil: [:vmThread| (self tryLockVMOwnerTo: vmThread index) ifTrue: [(self startThreadForThreadInfo: vmThread) ifFalse: [self releaseVM]]]! Item was changed: ----- Method: CogThreadManager>>ensureWillingThread (in category 'scheduling') ----- ensureWillingThread | willingThread newIndex | willingThread := self willingVMThread. willingThread ifNotNil: [^willingThread]. 1 to: numThreads do: [:index| + (self vmThreadAt: index) vmThreadState = CTMUninitialized ifTrue: - (self vmThreadAt: index) state ifNil: [self startThreadForThreadIndex: index. ^self vmThreadAt: index]]. self startThreadForThreadIndex: (newIndex := numThreads + 1). ^self vmThreadAt: newIndex! Item was changed: ----- Method: CogThreadManager>>highestPriorityThreadIfHigherThan:expectedMax: (in category 'public api') ----- highestPriorityThreadIfHigherThan: activePriority expectedMax: maxPriority "Answer the first vmThread waiting to acquire the VM that is of higher priority than activePriority, or answer nil if none. If there is a higher priority thread then set the coInterpreter's maxWaitingPriority to either the priority of the next highest priority vmThread, or to 0 if none is waiting." <returnTypeC: #'CogVMThread *'> | vmThread highest nextHighest | <var: #vmThread type: #'CogVMThread *'> <var: #highest type: #'CogVMThread *'> <var: #nextHighest type: #'CogVMThread *'> highest := nextHighest := nil. "To make this fair we could remember the last index at which we found the highest and start the search at the following index." 1 to: numThreads do: [:i| vmThread := threads at: i. + vmThread vmThreadState = CTMWantingOwnership ifTrue: - vmThread state = CTMWantingOwnership ifTrue: [self assert: vmThread priority <= maxPriority. highest isNil ifTrue: [highest := vmThread] ifFalse: [vmThread priority > highest priority ifTrue: [nextHighest := highest. highest := vmThread] ifFalse: [nextHighest isNil ifTrue: [nextHighest := vmThread] ifFalse: [vmThread priority > nextHighest priority ifTrue: [nextHighest := vmThread]]]]]]. highest isNil ifTrue: [^nil]. highest priority <= activePriority ifTrue: [^nil]. coInterpreter setMaxWaitingPriorityTo: (nextHighest isNil ifTrue: [0] ifFalse: [nextHighest priority]). ^highest! Item was changed: ----- Method: CogThreadManager>>populate:from:to: (in category 'thread set') ----- populate: vmThreadPointers from: start to: finish "Populate vmThreadPointers with vmThreads over the given range." <var: #vmThreadPointers type: #'CogVMThread **'> | nThreads vmThreads | <var: #vmThreads type: #'CogVMThread *'> <var: #vmThread type: #'CogVMThread *'> <inline: true> nThreads := finish - start + 1. vmThreads := self cCode: [self calloc: nThreads _: (self sizeof: CogVMThread)] inSmalltalk: [CArrayAccessor on: ((1 to: nThreads) collect: [:ign| CogVMThread new])]. vmThreads ifNil: [^false]. "Since 0 is not a valid index, in C we allocate one extra CogVMThread and use 1-relative indices." self cCode: [start = 1 ifTrue: [vmThreadPointers at: 0 put: nil]] inSmalltalk: []. start to: finish do: [:i| | vmThread | vmThread := self addressOf: (vmThreads at: i - start). + vmThread initializeThreadState. (self ioNewOSSemaphore: (self addressOf: vmThread osSemaphore put: [:sem| vmThread osSemaphore: sem])) ~= 0 ifTrue: [start to: i - 1 do: [:j| vmThread := self addressOf: (vmThreads at: j - start). self ioDestroyOSSemaphore: (self addressOf: vmThread osSemaphore)]. self free: vmThreads. ^false]. vmThreadPointers at: i put: vmThread. vmThread awolProcLength: AWOLProcessesIncrement. vmThread index: i. self cCode: [] inSmalltalk: [vmThread reenterThreadSchedulingLoop: ReenterThreadSchedulingLoop new]]. ^true! Item was changed: ----- Method: CogThreadManager>>registerVMThread: (in category 'scheduling') ----- registerVMThread: vmThread "Register the current thread in the set of threads, initializing the supplied vmThread and setting the thread's thread block index there-to." <var: #vmThread type: #'CogVMThread *'> <returnTypeC: #'CogVMThread *'> + self assert: ((vmThread vmThreadState = CTMInitializing + or: [vmThread vmThreadState = CTMWantingOwnership]) and: [vmThread osThread isNil]). - self assert: ((vmThread state = CTMInitializing - or: [vmThread state = CTMWantingOwnership]) and: [vmThread osThread isNil]). vmThread osThread: self ioCurrentOSThread. self ioSetThreadLocalThreadIndex: vmThread index. self assert: self ioGetThreadLocalThreadIndex = vmThread index. ^vmThread! Item was changed: ----- Method: CogThreadManager>>startThreadForThreadInfo: (in category 'scheduling') ----- startThreadForThreadInfo: vmThread <var: #vmThread type: #'CogVMThread *'> <inline: false> + self assert: vmThread vmThreadState = CTMUninitialized. + vmThread setVmThreadState: CTMInitializing. - self assert: vmThread state isNil. - vmThread state: CTMInitializing. "self cCode: '' inSmalltalk: [coInterpreter transcript cr; nextPutAll: 'starting VM thread '; print: vmThread index; flush. (thisContext home stackOfSize: 10) do: [:ctxt| coInterpreter transcript cr; print: ctxt; flush]]." (self ioNewOS: (self cCoerce: #startVMThread: to: 'void (*)(void*)') Thread: vmThread) = 0 ifTrue: [self ioTransferTimeslice. ^true]. memoryIsScarce := true. "self cCode: [coInterpreter print: 'ERVT failed to spawn so memory is scarce'; cr]" ^false! Item was changed: ----- Method: CogThreadManager>>startThreadSubsystem (in category 'public api') ----- startThreadSubsystem "Initialize the threading subsystem, aborting if there is an error." | vmThread | <inline: false> self assert: threads = nil. vmOSThread := self ioCurrentOSThread. numThreadsIncrement := (self ioNumProcessors max: 2) min: 16. (self growThreadInfosToAtLeast: numThreadsIncrement * 2) ifFalse: [self error: 'no memory to start thread system']. self atomic_store: (self addressOf: vmOwner) _: 1. vmThread := threads at: self getVMOwner. + vmThread setVmThreadState: CTMInitializing. - vmThread state: CTMInitializing. self registerVMThread: vmThread. + vmThread setVmThreadState: CTMAssignableOrInVM! - vmThread state: CTMAssignableOrInVM! Item was changed: ----- Method: CogThreadManager>>startVMThread: (in category 'scheduling') ----- startVMThread: vmThread "Start a VM thread that will attempt to acquire the VM and proceed to run the VM, taking processes from the runnable process queue." <returnTypeC: #void> <var: #vmThread type: #'CogVMThread *'> + self assert: vmThread vmThreadState = CTMInitializing. self registerVMThread: vmThread. + vmThread setVmThreadState: CTMAssignableOrInVM. - vmThread state: CTMAssignableOrInVM. coInterpreter threadSchedulingLoop: vmThread! Item was changed: ----- Method: CogThreadManager>>unregisterVMThread: (in category 'scheduling') ----- unregisterVMThread: vmThread "Unegister the current thread in the set of threads." <var: #vmThread type: #'CogVMThread *'> + self assert: (vmThread vmThreadState ~= CTMUninitialized + and: [vmThread vmThreadState ~= CTMInitializing]). - self assert: (vmThread state ~= nil - and: [vmThread state ~= CTMInitializing]). vmThread + initializeThreadState; - state: nil; osThread: nil. self ioSetThreadLocalThreadIndex: 0! Item was changed: ----- Method: CogThreadManager>>unusedThreadInfo (in category 'thread set') ----- unusedThreadInfo "Answer a pointer to an unused CogVMThread, growing the sequence if necessary." <returnTypeC: #'CogVMThread *'> | vmThread index | <var: #vmThread type: #'CogVMThread *'> 1 to: numThreads do: [:i| vmThread := threads at: i. self assert: vmThread index = i. + vmThread vmThreadState = CTMUninitialized ifTrue: - vmThread state isNil ifTrue: [^vmThread]]. index := numThreads + 1. (self growThreadInfosToAtLeast: numThreads + numThreadsIncrement) ifFalse: [^nil]. ^threads at: index! Item was changed: ----- Method: CogThreadManager>>waitForWork: (in category 'public api') ----- waitForWork: vmThread "Wait for work." <var: #vmThread type: #'CogVMThread *'> <returnTypeC: #void> + vmThread setVmThreadState: CTMAssignableOrInVM. - vmThread state: CTMAssignableOrInVM. self deny: (self vmOwnerIs: vmThread index). self ioWaitOnOSSemaphore: (self addressOf: vmThread osSemaphore)! Item was changed: ----- Method: CogThreadManager>>wakeVMThreadFor: (in category 'public api') ----- wakeVMThreadFor: index "Transfer the VM to the thread with index. Called from a thread that finds the highest priority runnable process is bound to the thread with index index." <returnTypeC: #void> + | vmThread threadState | - | vmThread | self assert: (self vmIsOwned and: [(self vmOwnerIs: index) not]). self assert: (index between: 1 and: numThreads). vmThread := threads at: index. "Instead of going through a #disownVM: call, directly set the new VM owner. This has the advantage of avoiding a race for the different threads to become the new VM owner. In Simulation, this means we need to simulate a thread-switch." self cCode: [] inSmalltalk: [ self saveRegisterStateForCurrentProcess. self loadOrInitializeRegisterStateFor: index]. self setVMOwner: index. + threadState := vmThread vmThreadState. + threadState = CTMUninitialized + ifTrue: [self startThreadForThreadInfo: vmThread] + ifFalse: + [self assert: ((threadState = CTMWantingOwnership + or: [threadState = CTMAssignableOrInVM]) + or: [threadState = CTMInitializing]). - vmThread state - ifNil: [self startThreadForThreadInfo: vmThread] - ifNotNil: - [self assert: ((vmThread state = CTMWantingOwnership - or: [vmThread state = CTMAssignableOrInVM]) - or: [vmThread state = CTMInitializing]). self ioSignalOSSemaphore: (self addressOf: vmThread osSemaphore)]. self ioTransferTimeslice! Item was changed: ----- Method: CogThreadManager>>willingVMThread (in category 'thread set') ----- willingVMThread "Answer a pointer to a live CogVMThread in any of the ``will do VM work'' states (other than the current owner if the VM is owned), or nil if none. Preferentially answer threads wanting ownership." <returnTypeC: #'CogVMThread *'> | thread threadWantingVM threadWilling | <inline: false> threadWantingVM := threadWilling := nil. 1 to: numThreads do: [:i| (self vmOwnerIs: i) ifFalse: [thread := threads at: i. + thread vmThreadState = CTMWantingOwnership ifTrue: - thread state = CTMWantingOwnership ifTrue: [(threadWantingVM isNil or: [threadWantingVM priority < thread priority]) ifTrue: [threadWantingVM := thread]]. + thread vmThreadState = CTMAssignableOrInVM ifTrue: - thread state = CTMAssignableOrInVM ifTrue: [(threadWilling isNil or: [threadWilling priority < thread priority]) ifTrue: [threadWilling := thread]]]]. ^threadWantingVM ifNil: [threadWilling]! Item was changed: ----- Method: CogVMThread class>>instVarNamesAndTypesForTranslationDo: (in category 'translation') ----- instVarNamesAndTypesForTranslationDo: aBinaryBlock "enumerate aBinaryBlock with the names and C type strings for the inst vars to include in a CogVMThread struct." self allInstVarNames do: [:ivn| aBinaryBlock value: ivn value: (ivn caseOf: { ['awolProcesses'] -> [{#sqInt. '[', CogThreadManager awolProcessesIncrement printString, ']'}]. ['cStackPointer'] -> [#usqIntptr_t]. ['cFramePointer'] -> [#usqIntptr_t]. ['osSemaphore'] -> ['sqOSSemaphore']. ['osThread'] -> ['sqOSThread']. + ['reenterThreadSchedulingLoop'] -> ['jmp_buf']. + ['state'] -> ['volatile atomic_int'] } - ['reenterThreadSchedulingLoop'] -> ['jmp_buf'] } otherwise: [#sqInt])]! Item was added: + ----- Method: CogVMThread>>initializeThreadState (in category 'initialize-release') ----- + initializeThreadState + "Unfortunately this cannot be inlined as Slang otherwise screws up the generation of the `atomic_store` call." + <inline: false> + "In comparision to #initialize, this is also called in C code to initialize the VMThread, not just in the Smalltalk simulation." + self cCode: [] inSmalltalk: [state := AtomicValue new]. + self atomic_store: (self addressOf: self state) _: CTMUninitialized.! Item was added: + ----- Method: CogVMThread>>setVmThreadState: (in category 'accessing') ----- + setVmThreadState: anInteger + "Unfortunately this cannot be inlined by Slang, as the generation otherwise screws + up the atomic_store operation." + <inline: false> + |currentState| + currentState := self vmThreadState. + currentState caseOf: { + [CTMUninitialized] -> [self assert: anInteger = CTMInitializing]. + } otherwise: []. + + "The actual meat of the operation. The previous checks are only for debugging." + self atomic_store: (self addressOf: self state) _: anInteger.! Item was changed: ----- Method: CogVMThread>>state (in category 'accessing') ----- state - "Answer the value of state" ^ state! Item was removed: - ----- Method: CogVMThread>>state: (in category 'accessing') ----- - state: anObject - "Set the value of state" - - ^state := anObject! Item was added: + ----- Method: CogVMThread>>vmThreadState (in category 'accessing') ----- + vmThreadState + "Unfortunately this cannot be inlined by Slang, as the generation otherwise screws + up the atomic_load operation." + <inline: false> + ^ self atomic_load: (self addressOf: self state)! Item was changed: ----- Method: ObjectMemory>>num32BitUnitsOf: (in category 'object access') ----- num32BitUnitsOf: objOop "Answer the number of 32-bit units in the given non-immediate object. N..B. Rounds down 8-bit units, so a 7 byte object has 1 32-bit unit. Does not adjust the size of contexts by stackPointer." ^(self numBytesOf: objOop) >> 2! Item was changed: SharedPool subclass: #VMThreadingConstants instanceVariableNames: '' + classVariableNames: 'AWOLProcessesIncrement CTMAssignableOrInVM CTMInitializing CTMUnavailable CTMUninitialized CTMWantingOwnership ThreadIdIndex ThreadIdShift' - classVariableNames: 'AWOLProcessesIncrement CTMAssignableOrInVM CTMInitializing CTMUnavailable CTMWantingOwnership ThreadIdIndex ThreadIdShift' poolDictionaries: '' category: 'VMMaker-Multithreading'! !VMThreadingConstants commentStamp: '<historical>' prior: 0! VMThreadingConstants ensureClassPool. CogThreadManager classPool keys do: [:k| VMThreadingConstants classPool declare: k from: CogThreadManager classPool]. CoInterpreterMT classPool keys do: [:k| VMThreadingConstants classPool declare: k from: CoInterpreterMT classPool].!

1 0

VM Maker: VMMaker.threaded-LM.3339.mcz
by commits＠source.squeak.org 03 Oct '23

03 Oct '23

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev October 2023