David T. Lewis uploaded a new version of System to project The Treated Inbox: http://source.squeak.org/treated/System-dtl.1447.mcz
==================== Summary ====================
Name: System-dtl.1447 Author: dtl Time: 18 February 2024, 10:30:07.309835 am UUID: 532176ff-7a46-4306-a373-252ca1cc09f2 Ancestors: System-codefrau.1446
Move the low space watcher from SmalltalkImage to new class LowSpaceWatcher having a singleton with responsibility for the low space semaphore and low space watcher process. Add class and method comments and split lowSpaceWatcher into smaller methods to clarify steps in the process. Add #registerCleaner for adding savvy memory hogs to the memory hog registry. For organization and documentation only, no functional change to the low space watcher or memory policies.
=============== Diff against System-codefrau.1446 ===============
Item was added: + Object subclass: #LowSpaceWatcher + instanceVariableNames: 'lowSpaceProcess lowSpaceSemaphore' + classVariableNames: 'Default MemoryHogs' + poolDictionaries: '' + category: 'System-Support'! + + !LowSpaceWatcher commentStamp: 'dtl 1/28/2024 17:56' prior: 0! + LowSpaceWatcher is responsible for responding to a notification from the virtual machine that memory is low and that it (the VM) may soon be unable to support additional object memory allocations. A single default instance for the image manages both the low space semaphore and the process that waits on the semaphore to handle low space conditions. When the low space watcher is notified that memory is low, it attempts to identify the process likely associated with the low space condition and to provide a notifier to allow the problem condition to be corrected. + + If a low space condition is detected in the virtual machine, the low space semaphore is signalled either directly by the VM or indirectly through an OutOfMemory error following a failed primitive invocation. The VM is responsible for identifying low space conditions, and its behavior will vary depending on both the VM implementation and the memory system of the underlying platform operating system. In particular, on a virtual machine operating system the VM may continue to receive memory allocations from the operating system while the operating system attempts to support the memory requests by increased swapping to disk. Under these conditions, the low space semaphore may not be signalled because system memory still appears to be available, even though performance is severely degraded due to swapping. + + The VM may provide control of its memory usage, typically through command line parameters or VM parameters that can be set from the image. These parameters will affect when and if the low space watcher is signalled by the VM. + + A registry is maintained in class variable MemoryHogs to identify objects (classes) know how to release unneeded memory when sent the message #freeSomeSpace. The low space watcher process will send this message when a low spaced condition is encountered. + + To signal the low space watcher and simulate a low space condition, evaluate "LowSpaceWatcher default signalLowSpace".!
Item was added: + ----- Method: LowSpaceWatcher class>>default (in category 'instance creation') ----- + default + "The singleton low space watcher." + + ^ Default ifNil: [ Default := self new start ] + !
Item was added: + ----- Method: LowSpaceWatcher class>>install (in category 'instance creation') ----- + install + "Start a process to watch for low-space conditions." + "Smalltalk installLowSpaceWatcher" + + self default stop; start + + !
Item was added: + ----- Method: LowSpaceWatcher class>>registerCleaner: (in category 'memory hog registry') ----- + registerCleaner: memoryCleaner + "Add memoryCleaner to the MemoryHog list where memoryCleaner is an object or class that responds to #freeSomeSpace." + + (memoryCleaner respondsTo: #freeSomeSpace) + ifFalse: [ ^ self error: memoryCleaner asString, ' does not understand #freeSomeSpace' ]. + (self default memoryHogs includes: memoryCleaner) + ifFalse: [ MemoryHogs add: memoryCleaner ].!
Item was added: + ----- Method: LowSpaceWatcher class>>signalLowSpace (in category 'signal low space') ----- + signalLowSpace + "Signal the low-space semaphore to alert the user that space is running low." + + ^ self default signalLowSpace.!
Item was added: + ----- Method: LowSpaceWatcher>>disableVirtualMachineNotifications (in category 'process steps') ----- + disableVirtualMachineNotifications + "Ask the virtual machine to stop sending low space interrupts, and forget the current semaphore and watcher process." + + self primSignalAtBytesLeft: 0. "disable low space interrupts" + self primLowSpaceSemaphore: nil. + lowSpaceProcess := nil. + !
Item was added: + ----- Method: LowSpaceWatcher>>findPreemptedProcess (in category 'process steps') ----- + findPreemptedProcess + "Find the process that was active at the time of the low space interrupt. Check + first if the virtual machine has recorded the process that was active at the time + that it detected the low space detection, otherwise answer the process that was + most recently preempted when the semaphore was signalled." + + | preemptedProcess | + preemptedProcess := (Smalltalk specialObjectsArray at: 23) + ifNil: [Processor preemptedProcess "if in-image signal of OutOfMemory"]. + Smalltalk specialObjectsArray at: 23 put: nil. + ^ preemptedProcess + + !
Item was added: + ----- Method: LowSpaceWatcher>>handleLowSpaceCondition (in category 'process') ----- + handleLowSpaceCondition + "A low space condition has been detected. Do whatever may be necessary to free + some memory in the image before proceeding." + + | preemptedProcess | + preemptedProcess := self findPreemptedProcess. + self notifyMemoryHogs ifFalse: [ + self logError: preemptedProcess. + self notifyUser: preemptedProcess ]. + + !
Item was added: + ----- Method: LowSpaceWatcher>>initializeVirtualMachineNotifications (in category 'process steps') ----- + initializeVirtualMachineNotifications + "Let the virtual machine know how to notify the image of low space conditions." + + Smalltalk specialObjectsArray at: 23 put: nil. "process causing low space will be saved here" + lowSpaceSemaphore := Semaphore new. + self primLowSpaceSemaphore: lowSpaceSemaphore. + self primSignalAtBytesLeft: Smalltalk lowSpaceThreshold. "enable low space interrupts" + !
Item was added: + ----- Method: LowSpaceWatcher>>logError: (in category 'process steps') ----- + logError: preemptedProcess + + Preferences logDebuggerStackToFile ifTrue: [ + Smalltalk + logError: 'Space is low' + inContext: preemptedProcess suspendedContext + to: 'LowSpaceDebug.log' ]. + !
Item was added: + ----- Method: LowSpaceWatcher>>lowSpaceChoices (in category 'memory space') ----- + lowSpaceChoices + "Return a notifier message string to be presented when space is running low." + + ^ 'Warning!! Squeak is almost out of memory!! + + Low space detection is now disabled. It will be restored when you close or proceed from this error notifier. Don''t panic, but do proceed with caution. + + Here are some suggestions: + + If you suspect an infinite recursion (the same methods calling each other again and again), then close this debugger, and fix the problem. + + If you want this computation to finish, then make more space available (read on) and choose "proceed" in this debugger. Here are some ways to make more space available... + > Close any windows that are not needed. + > Get rid of some large objects (e.g., images). + > Leave this window on the screen, choose "save as..." from the screen menu, quit, restart the Squeak VM with a larger memory allocation, then restart the image you just saved, and choose "proceed" in this window. + + If you want to investigate further, choose "debug" in this window. Do not use the debugger "fullStack" command unless you are certain that the stack is not very deep. (Trying to show the full stack will definitely use up all remaining memory if the low-space problem is caused by an infinite recursion!!). + + ' + !
Item was added: + ----- Method: LowSpaceWatcher>>lowSpaceWatcher (in category 'process') ----- + lowSpaceWatcher + "Wait until the low space semaphore is signalled, then take appropriate actions. This + method is run once in the low space watcher process. Before terminating, it schedules + a new process to handle the next low space condition." + + self safeToInstall ifFalse: [ ^ Beeper beep]. + self initializeVirtualMachineNotifications. + lowSpaceSemaphore wait. + self disableVirtualMachineNotifications; handleLowSpaceCondition. + self start. "schedule a new process to handle the next low space condition" + + !
Item was added: + ----- Method: LowSpaceWatcher>>lowSpaceWatcherProcess (in category 'process') ----- + lowSpaceWatcherProcess + "Answer the process in which lowSpaceWatcher is running. A process browser + can use this to identify and label the system wide low space watcher process." + ^lowSpaceProcess!
Item was added: + ----- Method: LowSpaceWatcher>>memoryHogs (in category 'memory space') ----- + memoryHogs + "Answer the list of objects to notify with #freeSomeSpace if memory gets full." + + ^ MemoryHogs ifNil: [MemoryHogs := OrderedCollection new]!
Item was added: + ----- Method: LowSpaceWatcher>>notifyMemoryHogs (in category 'process steps') ----- + notifyMemoryHogs + "Try to clean up memory hogs. Answer true if sufficient memory was released." + + | free | + self memoryHogs isEmpty + ifTrue: [ ^ false ] + ifFalse: [ free := Smalltalk bytesLeft. + self memoryHogs + do: [ :hog | hog freeSomeSpace ]. + ^ Smalltalk bytesLeft > free ] + + !
Item was added: + ----- Method: LowSpaceWatcher>>notifyUser: (in category 'process steps') ----- + notifyUser: preemptedProcess + "Interrupt the user to allow the low space condition to be corrected. The user + is now unprotected until the low space watcher is re-installed." + + Project current + interruptName: 'Space is low' + message: self lowSpaceChoices + preemptedProcess: preemptedProcess. + !
Item was added: + ----- Method: LowSpaceWatcher>>primLowSpaceSemaphore: (in category 'primitive access') ----- + primLowSpaceSemaphore: aSemaphore + "Primitive. Register the given Semaphore to be signalled when the + number of free bytes drops below some threshold. Disable low-space + interrupts if the argument is nil." + + <primitive: 124> + self primitiveFailed!
Item was added: + ----- Method: LowSpaceWatcher>>primSignalAtBytesLeft: (in category 'primitive access') ----- + primSignalAtBytesLeft: numBytes + "Tell the interpreter the low-space threshold in bytes. When the free + space falls below this threshold, the interpreter will signal the low-space + semaphore, if one has been registered. Disable low-space interrupts if the + argument is zero. Fail if numBytes is not an Integer." + + <primitive: 125> + self primitiveFailed!
Item was added: + ----- Method: LowSpaceWatcher>>safeToInstall (in category 'process steps') ----- + safeToInstall + "Free space must be above threshold before starting low space watcher" + + ^Smalltalk garbageCollectMost > Smalltalk lowSpaceThreshold + or: [Smalltalk garbageCollect > Smalltalk lowSpaceThreshold] + + !
Item was added: + ----- Method: LowSpaceWatcher>>signalLowSpace (in category 'memory space') ----- + signalLowSpace + "Signal the low-space semaphore to alert the user that space is running low." + + lowSpaceSemaphore signal.!
Item was added: + ----- Method: LowSpaceWatcher>>start (in category 'initialize-release') ----- + start + "Start a new low space watcher process that will register a semaphore with the + virtual machine and wait for the semaphore to be signalled if a low space condition + is detected." + + lowSpaceProcess := [self lowSpaceWatcher] forkAt: Processor lowIOPriority. + !
Item was added: + ----- Method: LowSpaceWatcher>>stop (in category 'initialize-release') ----- + stop. + "Ask the virtual machine to disable low space interrupts, then terminate the watcher process." + + self primSignalAtBytesLeft: 0. "disable low-space interrupts" + lowSpaceProcess == nil ifFalse: [lowSpaceProcess terminate]. + lowSpaceProcess := lowSpaceSemaphore := nil. + !
Item was changed: Object subclass: #SmalltalkImage instanceVariableNames: 'globals' + classVariableNames: 'EndianCache LastImageName LastQuitLogPosition LastStats PlatformNameCache ShutDownList SourceFileVersionString StartUpList StartupStamp VMMakerVersion WordSize' - classVariableNames: 'EndianCache LastImageName LastQuitLogPosition LastStats LowSpaceProcess LowSpaceSemaphore MemoryHogs PlatformNameCache ShutDownList SourceFileVersionString StartUpList StartupStamp VMMakerVersion WordSize' poolDictionaries: '' category: 'System-Support'!
!SmalltalkImage commentStamp: 'dtl 3/6/2010 14:00' prior: 0! I represent the current image and runtime environment, including system organization, the virtual machine, object memory, plugins and source files. My instance variable #globals is a reference to the system dictionary of global variables and class names.
My singleton instance is called Smalltalk.!
Item was changed: ----- Method: SmalltalkImage>>installLowSpaceWatcher (in category 'memory space') ----- installLowSpaceWatcher "Start a process to watch for low-space conditions." "Smalltalk installLowSpaceWatcher"
+ LowSpaceWatcher install - self primSignalAtBytesLeft: 0. "disable low-space interrupts" - LowSpaceProcess == nil ifFalse: [LowSpaceProcess terminate]. - LowSpaceProcess := [self lowSpaceWatcher] newProcess. - LowSpaceProcess priority: Processor lowIOPriority. - LowSpaceProcess resume.
!
Item was removed: - ----- Method: SmalltalkImage>>lowSpaceChoices (in category 'memory space') ----- - lowSpaceChoices - "Return a notifier message string to be presented when space is running low." - - ^ 'Warning!! Squeak is almost out of memory!! - - Low space detection is now disabled. It will be restored when you close or proceed from this error notifier. Don''t panic, but do proceed with caution. - - Here are some suggestions: - - If you suspect an infinite recursion (the same methods calling each other again and again), then close this debugger, and fix the problem. - - If you want this computation to finish, then make more space available (read on) and choose "proceed" in this debugger. Here are some ways to make more space available... - > Close any windows that are not needed. - > Get rid of some large objects (e.g., images). - > Leave this window on the screen, choose "save as..." from the screen menu, quit, restart the Squeak VM with a larger memory allocation, then restart the image you just saved, and choose "proceed" in this window. - - If you want to investigate further, choose "debug" in this window. Do not use the debugger "fullStack" command unless you are certain that the stack is not very deep. (Trying to show the full stack will definitely use up all remaining memory if the low-space problem is caused by an infinite recursion!!). - - ' - !
Item was removed: - ----- Method: SmalltalkImage>>lowSpaceWatcher (in category 'memory space') ----- - lowSpaceWatcher - "Wait until the low space semaphore is signalled, then take appropriate actions." - - | free preemptedProcess | - self garbageCollectMost <= self lowSpaceThreshold - ifTrue: [self garbageCollect <= self lowSpaceThreshold - ifTrue: ["free space must be above threshold before - starting low space watcher" - ^ Beeper beep]]. - - Smalltalk specialObjectsArray at: 23 put: nil. "process causing low space will be saved here" - LowSpaceSemaphore := Semaphore new. - self primLowSpaceSemaphore: LowSpaceSemaphore. - self primSignalAtBytesLeft: self lowSpaceThreshold. "enable low space interrupts" - - LowSpaceSemaphore wait. "wait for a low space condition..." - - self primSignalAtBytesLeft: 0. "disable low space interrupts" - self primLowSpaceSemaphore: nil. - LowSpaceProcess := nil. - - "The process that was active at the time of the low space interrupt." - preemptedProcess := (Smalltalk specialObjectsArray at: 23) - ifNil: [Processor preemptedProcess "if in-image signal of OutOfMemory"]. - Smalltalk specialObjectsArray at: 23 put: nil. - - "Note: user now unprotected until the low space watcher is re-installed" - - self memoryHogs isEmpty - ifFalse: [free := self bytesLeft. - self memoryHogs - do: [ :hog | hog freeSomeSpace ]. - self bytesLeft > free - ifTrue: [ ^ self installLowSpaceWatcher ]]. - - Preferences logDebuggerStackToFile ifTrue: [ - self - logError: 'Space is low' - inContext: preemptedProcess suspendedContext - to: 'LowSpaceDebug.log']. - - Project current - interruptName: 'Space is low' - message: self lowSpaceChoices - preemptedProcess: preemptedProcess - !
Item was changed: ----- Method: SmalltalkImage>>lowSpaceWatcherProcess (in category 'memory space') ----- lowSpaceWatcherProcess + ^LowSpaceWatcher default ifNotNil: [ :watcher | watcher lowSpaceWatcherProcess ]! - ^LowSpaceProcess!
Item was removed: - ----- Method: SmalltalkImage>>memoryHogs (in category 'memory space') ----- - memoryHogs - "Answer the list of objects to notify with #freeSomeSpace if memory gets full." - - ^ MemoryHogs ifNil: [MemoryHogs := OrderedCollection new]!
Item was removed: - ----- Method: SmalltalkImage>>primLowSpaceSemaphore: (in category 'memory space') ----- - primLowSpaceSemaphore: aSemaphore - "Primitive. Register the given Semaphore to be signalled when the - number of free bytes drops below some threshold. Disable low-space - interrupts if the argument is nil." - - <primitive: 124> - self primitiveFailed!
Item was removed: - ----- Method: SmalltalkImage>>primSignalAtBytesLeft: (in category 'memory space') ----- - primSignalAtBytesLeft: numBytes - "Tell the interpreter the low-space threshold in bytes. When the free - space falls below this threshold, the interpreter will signal the low-space - semaphore, if one has been registered. Disable low-space interrupts if the - argument is zero. Fail if numBytes is not an Integer." - - <primitive: 125> - self primitiveFailed!
Item was removed: - ----- Method: SmalltalkImage>>signalLowSpace (in category 'memory space') ----- - signalLowSpace - "Signal the low-space semaphore to alert the user that space is running low." - - LowSpaceSemaphore signal.!