Hi Marcel,

On Mon, Jul 11, 2022 at 11:15 PM Marcel Taeumel <marcel.taeumel at hpi.de>
wrote:

> Aha. Via #handleFailingFailingBasicNew:, OutOfMemory >> #defaultAction
> signals the LowSpaceSemaphore. And at that point, special object 23 was not
> yet set. Hmm....
>

  the VM sets Smalltalk specialObjects at: 23 and signals the low space
semaphore when it detects that heap space is low.  This is complicated, but
essentially Smalltalk will have asked for some allocation to be done, and
typically only a large allocation will trip the low space limit, but a
cumulative series of small allocations *that are held onto and not
reclaimable via a scavenge* can also trip the level.  A flag in the vm
(needGCFlag) will get set, and  the VM's "you need to respond to events"
flag gets set by the allocation that failed and/or by the allocation that
tripped the new space max size threshold.  Then, in checking for events,
the VM runs the garbage collector (the scavenger).  At this point it may
see that available space is low and then there is a fork in the road.

If there is no old space size limit in effect, the VM will try and grow old
space by adding a new segment.  This is a path that leads quickly to the VM
growing memory to a level where the system swaps itself into
unresponsiveness.

If there is a size limit in effect then the VM won't be able to grow old
space, will find that the total ammount of free space is below the low
space threshold, and will signal the low space semaphore and set Smalltalk
specialObjects at: 23 to the process that is active.

Sorry this is so complicated; it "was always like this".

Executive summary, test the system with an old space size limit in place
(Smalltalk vmParameterAt: 67, look for "#67 0 the maximum allowed size of
old space (if zero there is no limit)" in the About Squeak->VM Parameters
tab in the SystermReporter).  Then, if that doesn't work, come back and
complain.  If it does work then we have some documenting to do to explain
how this fragile and complex mechanism can work "if it's Thursday, in the
winter months, and there are clouds in the west".

'nuff said?

>
> Best,
> Marcel
>
> Am 12.07.2022 08:13:15 schrieb Marcel Taeumel <marcel.taeumel at hpi.de>:
> Hi Eliot --
>
> > Our #lowSpaceWatcher seems to be broken because "Smalltalk
> specialObjectsArray at: 23" is always "nil".
> >> [...] The vm sets it to the active process when it signals the low
> space semaphore.
>
> And that's not working at the moment. Even after the LowSpaceSemaphore got
> signaled:
>
> Smalltalk vmParameterAt: 67 put: 1 * 1024 * 1024 * 1024.
> Array new: (0.125 * 1 * 1024 * 1024 * 1024) rounded.
>
> One has to push CMD+Dot to then see that the #lowSpaceWatcher complains
> that  it cannot retrieve the preemptedProcess via special object 23.
>
> > Has #primSignalAtBytesLeft: (primitive 125) any effect these days?
> >> Yes.
>
> Hmm... I can only find places that set it to 0. :-) ... Wait ... Ah!
> Nevermind. xD
>
> >> So infinite recursion (as it always did) causes heap growth, and the
> issue of sluggishness did to paging comes to the fore.
>
> Ah! That was my hypothesis. Yet, since the #lowSpaceWatcher seems to be
> broken, I wondered what was going on.
>
> Thanks! :-)
>
> Best,
> Marcel
>
>
> Am 12.07.2022 08:02:28 schrieb Eliot Miranda <eliot.miranda at gmail.com>:
> Hi Marcel,
>
> On Jul 11, 2022, at 6:15 AM, Marcel Taeumel <marcel.taeumel at hpi.de> wrote:
>
>
> 
> Hi all --
>
> What's the current state of our low-space guards in Squeak? That
> OutOfMemory error only works with larger allocations, right?
>
> Our #lowSpaceWatcher seems to be broken because "Smalltalk
> specialObjectsArray at: 23" is always "nil". Should we replace it with
> "Processor preemptedProcecss"?
>
>
> No, it *should* be nil.  The vm sets it to the active process when it
> signals the low space semaphore.  Hence "Smalltalk specialObjectsArray
> at: 23" reliably informs the image level low space code which process
> performed the allocation that tripped the low space trigger.
>
> Has #lowSpaceThreshold any effect these days?
>
>
> Yes.  But given that Spur attempts to grow the heap dynamically and
> contemporary OSs are only too happy to comply, the low space trigger may
> only be seen to work if a limit on old space size has been installed.
> Otherwise it’ll most likely not trigger before the system starts to page
> itself into sluggishness.
>
>
> Has #primSignalAtBytesLeft: (primitive 125) any effect these days?
>
>
> Yes.
>
>
> Can there ever be an image-level protection against stack-page shortage?
> VM parameter 42 seems to be 50 (num stack pages), which lasts for quite
> some time in endless recursion ... but CMD+Dot stops working at some point
> because of some stack-growing issues? And then the VM crashes eventually.
>
>
> Again that’s not how things work.  The stack zone is fixed size.  When a
> new stack page is needed the oldest page is evacuated to the heap in the
> form of Context objects.  So infinite recursion (as it always did) causes
> heap growth, and the issue of sluggishness did to paging comes to the fore.
>
>
>
> Best,
> Marcel
>
>
>

-- 
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20220712/b8a77c01/attachment.html>