Marcel Taeumel uploaded a new version of Kernel to project The Treated Inbox:
http://source.squeak.org/treated/Kernel-jar.1480.mcz
==================== Summary ====================
Name: Kernel-jar.1480
Author: jar
Time: 9 June 2022, 7:53:32.496688 pm
UUID: 1504c554-482d-a94e-b292-ab2c46c6d865
Ancestors: Kernel-mt.1479
Fix a bug: when debugging things like this:
[^2] ensure: [Transcript cr; show: 'done']
if we step into the protected block [^2] and then step over ^2, we incorrectly get a BlockCannotReturn error.
Improved comment; please remove Kernel-jar.1415 from the Inbox.
This is an alternative to my older version in Kernel-jar.1421; the solution remains the same but it attempts to present a cleaner code (addressing Christoph's objection in [2] and [3]) and improves the comment.
The bug is described in detail in Kernel-nice.1407 and discussed in [1] and most recently in [2] and [3]:
[1] http://forum.world.st/stepping-over-non-local-return-in-a-protected-block-t…
[2] http://lists.squeakfoundation.org/pipermail/squeak-dev/2021-August/216214.h…
[3] http://lists.squeakfoundation.org/pipermail/squeak-dev/2021-November/216971…
=============== Diff against Kernel-mt.1479 ===============
Item was changed:
----- Method: Context>>resume:through: (in category 'controlling') -----
resume: value through: firstUnwindCtxt
"Unwind thisContext to self and resume with value as result of last send.
Execute any unwind blocks while unwinding.
ASSUMES self is a sender of thisContext."
| ctxt unwindBlock |
self isDead ifTrue: [self cannotReturn: value to: self].
+ ctxt := firstUnwindCtxt value. "evaluate in case firstUnwindCtxt is a block (see comment in #return:from:)"
- ctxt := firstUnwindCtxt.
[ctxt isNil] whileFalse:
[(ctxt tempAt: 2) ifNil:
[ctxt tempAt: 2 put: true.
unwindBlock := ctxt tempAt: 1.
thisContext terminateTo: ctxt.
unwindBlock value].
ctxt := ctxt findNextUnwindContextUpTo: self].
thisContext terminateTo: self.
^value
!
Item was changed:
----- Method: Context>>return:from: (in category 'instruction decoding') -----
return: value from: aSender
"For simulation. Roll back self to aSender and return value from it. Execute any unwind blocks on the way. ASSUMES aSender is a sender of self"
+ "Note: when debugging things like this:
+ [^2] ensure: [Transcript cr; show: 'done']
+ if we step into the protected block [^2] and then step over ^2, we incorrectly get a BlockCannotReturn.
+ The root cause of this is that during simulation #runUntilErrorOrReturnFrom: inserts a new unwind
+ context between the top of the stack and 'newTop' so we can't supply the next unwind context to
+ #aboutToReturn:through: now but must postpone the search for the next unwind context until the
+ right moment, i.e. until #resume:through: gets executed using the supplied next unwind context.
+ One solution is to send a block
+ [thisContext findNextUnwindContextUpTo: newTop]
+ that will execute the search in #resume:through: instead of sending the next unwind context.
+ Indeed we must modify #resume:through: to evaluate its second argument in case it's this block.
+
+ The objection against such a solution is we are creating a dependency between these two methods.
+ See more in http://lists.squeakfoundation.org/pipermail/squeak-dev/2021-August/216214.h…"
+
| newTop |
aSender isDead ifTrue:
[^self send: #cannotReturn: to: self with: {value}].
newTop := aSender sender.
(self findNextUnwindContextUpTo: newTop) ifNotNil:
+ [^self send: #aboutToReturn:through: to: self with: {value. [thisContext findNextUnwindContextUpTo: newTop]}].
- [:unwindProtectCtxt|
- ^self send: #aboutToReturn:through: to: self with: {value. unwindProtectCtxt}].
self releaseTo: newTop.
newTop ifNotNil: [newTop push: value].
^newTop!
Marcel Taeumel uploaded a new version of Kernel to project The Treated Inbox:
http://source.squeak.org/treated/Kernel-jar.1498.mcz
==================== Summary ====================
Name: Kernel-jar.1498
Author: jar
Time: 30 January 2023, 5:33:16.106677 pm
UUID: f76b2d65-c441-1a44-a925-e7c1e0d45cbb
Ancestors: Kernel-tpr.1497
fix four currently failing tests; improve unwind's behavior in #critical sections and #terminate resiliency in case of accidental preemption; fix a bug in #unwindAndStop:, add more tests and comments
complemented by KernelTests-jar.443
please kindly remove KernelTests-jar.437 and Kernel-jar.1487 from the Inbox.
=============== Diff against Kernel-tpr.1497 ===============
Item was added:
+ ----- Method: Context>>unwindAndStop: (in category 'private') -----
+ unwindAndStop: aProcess
+ "I'm a helper method to #terminate; I create and answer
+ a helper stack for a terminating process to unwind itself from.
+ Note: push a fake return value to create a proper top context."
+
+ ^(self class contextEnsure: [self unwindTo: nil])
+ privSender: [aProcess suspend] asContext;
+ push: nil
+ !
Item was changed:
----- Method: Process>>suspendAndReleaseCriticalSection (in category 'private') -----
suspendAndReleaseCriticalSection
"Figure out if we are terminating a process that is in the ensure: block of a critical section.
If it hasn't made progress but is beyond the wait (which we can tell by the oldList being
+ one of the runnable lists, i.e. a LinkedList, not a Semaphore or Mutex), then the ensure:
- one of the runnable lists, i.e. a LinkedList, not a Semaphore or Mutex, et al), then the ensure:
block needs to be run."
| oldList selectorJustSent |
+ "Suspend and unblock the receiver from a condition variable using the suspend primitive #88.
- "Suspend and unblock the receiver from a condition variable using suspend primitive #88.
It answers the list the receiver was on before the suspension."
oldList := self suspendAndUnblock.
+ suspendedContext ifNotNil: [:context |
+ (context method pragmaAt: #criticalSection) ifNil: [^self].
+ "If still blocked at the condition variable of a critical section, skip the rest of the current context."
+ (oldList isNil or: [oldList class == LinkedList]) ifFalse: [
+ suspendedContext := context pc: context endPC.
+ ^self].
+ "Now we are somewhere beyond the wait/lock primitive, figure out where, and if necessary,
+ step into the ensure block to allow the unwind block to be run."
+ selectorJustSent := context selectorJustSentOrSelf.
- (oldList isNil or: [oldList class == LinkedList]) ifFalse: [^self].
+ "If still haven't made progress beyond the wait then the ensure block has not been activated,
+ so step into it."
+ selectorJustSent == #wait ifTrue: [
+ suspendedContext := context stepToCallee].
- ((suspendedContext ifNil: [^self]) method pragmaAt: #criticalSection) ifNil: [^self].
- selectorJustSent := suspendedContext selectorJustSentOrSelf.
+ "If still haven't made progress beyond the lock primitive and the lock primitive just acquired ownership
+ (indicated by it answering false) then the ensure block has not been activated, so step into it."
+ (selectorJustSent == #primitiveEnterCriticalSection
+ or: [selectorJustSent == #primitiveTestAndSetOwnershipOfCriticalSection]) ifTrue: [
+ (context stackPtr > 0 and: [context top == false]) ifTrue: [
+ suspendedContext := context stepToCallee]]]!
- "If still at the wait the ensure: block has not been activated, so signal to restore."
- selectorJustSent == #wait ifTrue:
- [suspendedContext receiver signal].
-
- "If still at the lock primitive and the lock primitive just acquired ownership (indicated by it answering false)
- then the ensure block has not been activated, so explicitly primitiveExitCriticalSection to unlock."
- (selectorJustSent == #primitiveEnterCriticalSection
- or: [selectorJustSent == #primitiveTestAndSetOwnershipOfCriticalSection]) ifTrue:
- [(suspendedContext stackPtr > 0
- and: [suspendedContext top == false]) ifTrue:
- [suspendedContext receiver primitiveExitCriticalSection]]!
Item was changed:
----- Method: Process>>terminate (in category 'changing process state') -----
+ terminate
+ "Stop the process that the receiver represents forever. Allow all pending unwind
+ blocks to run before terminating; if they are currently in progress, let them finish."
- terminate
- "Stop the process that the receiver represents forever.
- Unwind to execute pending #ensure:/#ifCurtailed: blocks before terminating;
- allow all unwind blocks to run; if they are currently in progress, let them finish."
+ "Note: This is the kind of behavior we expect when terminating a healthy process.
- "This is the kind of behavior we expect when terminating a healthy process.
See further comments in #terminateAggressively and #destroy methods dealing
with process termination when closing the debugger or after a catastrophic failure.
If terminating the active process, create a new stack and run unwinds from there.
If terminating a suspended process (including runnable and blocked), always
+ suspend the terminating process first so it doesn't accidentally get woken up,
+ and nil the suspended context to prevent accidental resumption or termination
+ while manipulating the suspended context.
+
- suspend the terminating process first so it doesn't accidentally get woken up.
Equally important is the side effect of the suspension: In 2022 a new suspend
semantics has been introduced: the revised #suspend backs up a process waiting
on a conditional variable to the send that invoked the wait state, while the previous
#suspend simply removed the process from the conditional variable's list it was
previously waiting on; see #suspend and #suspendAndUnblock comments.
+
+ If the process is blocked, waiting to access the #critical: section, release it properly.
-
- If the process is in the middle of a #critical: critical section, release it properly.
To allow a suspended process to unwind itself, create a new stack for the process
being terminated and resume the suspended process to complete its termination
+ from the new, parallel stack. Use a semaphore to make the process that invoked
- from the new parallel stack. Use a semaphore to make the process that invoked
the termination wait for self's completion. Execute the termination in the ensure
argument block to ensure it completes even if the terminator process itself gets
terminated before it's finished; see testTerminateInTerminate and others."
- | context |
self isActiveProcess ifTrue: [
+ ^(thisContext unwindAndStop: self) jump].
- context := thisContext.
- ^[[] ensure: [context unwindTo: nil]. self suspend] asContext jump].
+ [] ensure: [
- [] ensure: [ | terminator |
self suspendAndReleaseCriticalSection.
+ suspendedContext ifNotNil: [:context | | terminator |
+ suspendedContext := nil.
+ terminator := Semaphore new.
+ context bottomContext insertSender: (Context contextEnsure: [terminator signal]).
+ self priority: Processor activePriority;
+ suspendedContext: (context unwindAndStop: self);
+ resume.
+ terminator wait]]!
- context := suspendedContext ifNil: [^self].
- terminator := Semaphore new.
- context bottomContext insertSender: (Context contextEnsure: [terminator signal]).
- self suspendedContext: [[] ensure: [context unwindTo: nil]. self suspend] asContext;
- priority: Processor activePriority;
- resume.
- terminator wait]!
Marcel Taeumel uploaded a new version of Tools to project The Trunk:
http://source.squeak.org/trunk/Tools-mt.1189.mcz
==================== Summary ====================
Name: Tools-mt.1189
Author: mt
Time: 15 February 2023, 4:29:53.702748 pm
UUID: 9f8065c3-3843-aa49-b942-6e2dd5520b60
Ancestors: Tools-mt.1188
Avoid hiding errors in MessageTally code. Our current debugger is able to handle this case correctly.
=============== Diff against Tools-mt.1188 ===============
Item was changed:
----- Method: Debugger>>initialize (in category 'initialize-release') -----
initialize
super initialize.
- Smalltalk at: #MessageTally ifPresentAndInMemory: [ :tally |
- tally terminateTimerProcess].
-
selectingPC := true.
contextStackIndex := 0.
"The default termination procedure is aggressive to ignore currently running, and thus erroneous, ensure-blocks in the debugged process. The preference can change that."
terminateProcessSelector := self class closeMeansAbandon
ifTrue: [#terminateAggressively]
ifFalse:[#terminate].!
Marcel Taeumel uploaded a new version of Tools to project The Treated Inbox:
http://source.squeak.org/treated/Tools-jar.1161.mcz
==================== Summary ====================
Name: Tools-jar.1161
Author: jar
Time: 9 June 2022, 1:57:46.549529 pm
UUID: f3921b2b-5628-104d-98a9-1e99671344a2
Ancestors: Tools-ct.1160
Fix an inconsistent Debugger behavior WRT the new alternative abandon/terminate/destroy modes introduced by Kernel-ct.1434 and Tools-ct.1092.
The suggestion is to make all three termination methods equal in terms of how the chosen termination mode is propagated to #windowIsClosing. Current asymmetrical approach leads to an incorrect debugger termination via #terminate mode.
Replace termination of the debugged process with an indirect termination via a forked process (otherwise the debugger waits until the debugged process is terminated but if another error is raised during unwind the termination is halted and the assertion fail).
Following up a conversation in http://lists.squeakfoundation.org/pipermail/squeak-dev/2022-May/220675.html
Complemented by ToolsTests-jar.111 fixing the failing/incorrects tests in ToolsTests-ct.107
Please review as I'm no debugger expert and this is just my suggestion.
Superseeds (or replaces) Tools-jar.1159 (please kindly remove from Inbox)
=============== Diff against Tools-ct.1160 ===============
Item was changed:
CodeHolder subclass: #Debugger
+ instanceVariableNames: 'interruptedProcess contextStack contextStackIndex contextStackList receiverInspector receiverInspectorState contextVariablesInspector contextVariablesInspectorState externalInterrupt proceedValue selectingPC savedCursor isolationHead failedProject labelString message untilExpression terminationMethod'
- instanceVariableNames: 'interruptedProcess contextStack contextStackIndex contextStackList receiverInspector receiverInspectorState contextVariablesInspector contextVariablesInspectorState externalInterrupt proceedValue selectingPC savedCursor isolationHead failedProject labelString message untilExpression'
classVariableNames: 'ContextStackKeystrokes ErrorReportServer FullStackSize InterruptUIProcessIfBlockedOnErrorInBackgroundProcess NotifierStackSize SavedExtent StackSizeLimit WantsAnnotationPane'
poolDictionaries: ''
category: 'Tools-Debugger'!
!Debugger commentStamp: 'mt 12/17/2019 12:19' prior: 0!
I represent the machine state at the time of an interrupted process. I also represent a query path into the state of the process. The debugger is typically viewed through a window that views the stack of suspended contexts, the code for, and execution point in, the currently selected message, and inspectors on both the receiver of the currently selected message, and the variables in the current context.
Special note on recursive errors:
Some errors affect Squeak's ability to present a debugger. This is normally an unrecoverable situation. However, if such an error occurs in an isolation layer, Squeak will attempt to exit from the isolation layer and then present a debugger. Here is the chain of events in such a recovery.
* A recursive error is detected.
* The current project is queried for an isolationHead
* Changes in the isolationHead are revoked
* The parent project of isolated project is returned to
* The debugger is opened there and execution resumes.
If the user closes that debugger, execution continues in the outer project and layer. If, after repairing some damage, the user proceeds from the debugger, then the isolationHead is re-invoked, the failed project is re-entered, and execution resumes in that world.
---
In September 2019, we added MorphicDebugger and MVCDebugger to untangle framework-specific features in our debugger infrastructure. However, this is just an intermediate step. The overall goal would be to remove those two subclasses again while preserving their functionality. Mostly, MVC and Morphic differ in their GUI-process management. This means that "proceed" and "close" work differently depending on the process that is being debugged. --- One idea is to attach that framework-specific information to the process objects. See Process >> #environmentAt: and #environmentAt:put:. Also see ToolSet's #handle* and #debug* methods.!
Item was changed:
----- Method: Debugger>>abandon (in category 'context stack menu') -----
abandon
+ "close the debugger and terminate the debugged process"
- "abandon the debugger from its pre-debug notifier"
+ terminationMethod := #terminateAggressively.
+ self close
+ !
- self close.!
Item was changed:
----- Method: Debugger>>initialize (in category 'initialize') -----
initialize
super initialize.
Smalltalk at: #MessageTally ifPresentAndInMemory: [ :tally |
tally terminateTimerProcess].
externalInterrupt := false.
selectingPC := true.
+ contextStackIndex := 0.
+
+ terminationMethod := #terminateAggressively "debugger's default termination method"!
- contextStackIndex := 0.!
Item was changed:
----- Method: Debugger>>terminateProcess (in category 'context stack menu') -----
terminateProcess
+ "close the debugger and terminate the debugged process"
+ terminationMethod := #terminate.
+ self close
+ !
- interruptedProcess terminate.
- interruptedProcess := nil.
- self close.!
Item was changed:
----- Method: Debugger>>windowIsClosing (in category 'initialize') -----
windowIsClosing
"My window is being closed; clean up. Restart the low space watcher."
contextStack := nil.
receiverInspector := nil.
contextVariablesInspector := nil.
interruptedProcess == nil ifTrue: [^ self].
+ [interruptedProcess perform: terminationMethod.
+ interruptedProcess := nil] fork.
- interruptedProcess terminateAggressively.
- interruptedProcess := nil.
Smalltalk installLowSpaceWatcher. "restart low space handler"!
Marcel Taeumel uploaded a new version of Tools to project The Treated Inbox:
http://source.squeak.org/treated/Tools-jar.1159.mcz
==================== Summary ====================
Name: Tools-jar.1159
Author: jar
Time: 29 May 2022, 5:12:52.367993 pm
UUID: bed7f2b8-2dd3-9b45-8262-ff51bc9dd5c3
Ancestors: Tools-mt.1158
Fix an inconsistent Debugger behavior WRT the two alternative abandon/termination modes introduced by Kernel-ct.1434 and Tools-ct.1092
Following up a conversation in http://lists.squeakfoundation.org/pipermail/squeak-dev/2022-May/220675.html
Complemented by ToolsTests-jar.110 fixing the failing/incorrects tests in ToolsTests-ct.107
=============== Diff against Tools-mt.1158 ===============
Item was changed:
CodeHolder subclass: #Debugger
+ instanceVariableNames: 'interruptedProcess contextStack contextStackIndex contextStackList receiverInspector receiverInspectorState contextVariablesInspector contextVariablesInspectorState externalInterrupt proceedValue selectingPC savedCursor isolationHead failedProject labelString message untilExpression terminationMethod'
- instanceVariableNames: 'interruptedProcess contextStack contextStackIndex contextStackList receiverInspector receiverInspectorState contextVariablesInspector contextVariablesInspectorState externalInterrupt proceedValue selectingPC savedCursor isolationHead failedProject labelString message untilExpression'
classVariableNames: 'ContextStackKeystrokes ErrorReportServer FullStackSize InterruptUIProcessIfBlockedOnErrorInBackgroundProcess NotifierStackSize SavedExtent StackSizeLimit WantsAnnotationPane'
poolDictionaries: ''
category: 'Tools-Debugger'!
!Debugger commentStamp: 'mt 12/17/2019 12:19' prior: 0!
I represent the machine state at the time of an interrupted process. I also represent a query path into the state of the process. The debugger is typically viewed through a window that views the stack of suspended contexts, the code for, and execution point in, the currently selected message, and inspectors on both the receiver of the currently selected message, and the variables in the current context.
Special note on recursive errors:
Some errors affect Squeak's ability to present a debugger. This is normally an unrecoverable situation. However, if such an error occurs in an isolation layer, Squeak will attempt to exit from the isolation layer and then present a debugger. Here is the chain of events in such a recovery.
* A recursive error is detected.
* The current project is queried for an isolationHead
* Changes in the isolationHead are revoked
* The parent project of isolated project is returned to
* The debugger is opened there and execution resumes.
If the user closes that debugger, execution continues in the outer project and layer. If, after repairing some damage, the user proceeds from the debugger, then the isolationHead is re-invoked, the failed project is re-entered, and execution resumes in that world.
---
In September 2019, we added MorphicDebugger and MVCDebugger to untangle framework-specific features in our debugger infrastructure. However, this is just an intermediate step. The overall goal would be to remove those two subclasses again while preserving their functionality. Mostly, MVC and Morphic differ in their GUI-process management. This means that "proceed" and "close" work differently depending on the process that is being debugged. --- One idea is to attach that framework-specific information to the process objects. See Process >> #environmentAt: and #environmentAt:put:. Also see ToolSet's #handle* and #debug* methods.!
Item was changed:
----- Method: Debugger>>abandon (in category 'context stack menu') -----
abandon
+ "close the debugger and terminate the debugged process"
- "abandon the debugger from its pre-debug notifier"
+ terminationMethod := #terminateAggressively.
+ self close
+ !
- self close.!
Item was changed:
----- Method: Debugger>>initialize (in category 'initialize') -----
initialize
super initialize.
Smalltalk at: #MessageTally ifPresentAndInMemory: [ :tally |
tally terminateTimerProcess].
externalInterrupt := false.
selectingPC := true.
+ contextStackIndex := 0.
+
+ terminationMethod := #terminateAggressively "debugger's default termination method"!
- contextStackIndex := 0.!
Item was changed:
----- Method: Debugger>>terminateProcess (in category 'context stack menu') -----
terminateProcess
+ "close the debugger and terminate the debugged process"
+ terminationMethod := #terminate.
+ self close
+ !
- interruptedProcess terminate.
- interruptedProcess := nil.
- self close.!
Item was changed:
----- Method: Debugger>>windowIsClosing (in category 'initialize') -----
windowIsClosing
"My window is being closed; clean up. Restart the low space watcher."
contextStack := nil.
receiverInspector := nil.
contextVariablesInspector := nil.
interruptedProcess == nil ifTrue: [^ self].
+ interruptedProcess perform: terminationMethod.
- interruptedProcess terminateAggressively.
interruptedProcess := nil.
Smalltalk installLowSpaceWatcher. "restart low space handler"!