Hi Eliot,

I'm finally starting to understand :)
The real reason why we need #terminateTo: in the image is the implementation of StackInterpreter >> #commonReturn.
There are two non-trivial scenarios:
1) home is on the sender chain and no unwind context is found -> then just return internally inside the VM and (!) mark all intervening contexts/pages as returned from (nil pc and sender); this is effectively as if executing terminateTo.
2) home is on the sender chain and an unwind context's found -> then the vm just sends aboutToReturn and lets the image deal with the unwinds.
And this is, in my opinion, the real reason why we need to call #terminateTo: *in the image*, to make the two scenarios behave consistently, because otherwise the result of a return would depend on the presence/absence of unwind blocks.

Example:
Define a test method:

Object >> testMe
                [^[^2]] ensure: []

and inspect 'self testMe'
--> the returned closure will have a *dead* outerContext as expected.

Now, in Context >> #resumeEvaluating: aBlock through: firstUnwindCtxtOrNil replace
'thisContext terminateTo: self.'
with
'thisContext privSender: self.'

and inspect 'self testMe'
--> the returned closure will have the outerContext *very much alive*.

If you define the test method *without* the ensure:

Object >> testMe
                ^[^2]

Then in both cases 'self testMe' will return a closure with a *dead* outerContext.

And this inconsistency is bad indeed.


> Let’s imagine that somewhere along the chain between the receiver and argument of terminateTo: a block with a up arrow return was created whose home was on that chain.  Were the contexts not marked as having been returned from then a subsequent activation of the block might actually be able to return.

Yes! here's an example:

testMe
                self returnTwo: [:returnTwo | [^returnTwo] ensure: []]

returnTwo: aBlock
                aBlock value: [^2]

If you do 'self testMe value'
then for #teminateTo: replaced with #privSender: (in #resumeEvaluating:through: as in the previous example) the computation will subsequently activate the inside block [^2] and return from it too without any accident. (Note: for some reason when debugging, the computation returns from both blocks and then nicely continues but when running normally, it returns from both blocks as in simulation but then raises a cannot return error later, from another return; apparently, the VM doesn't like something there while the debugger sees no problem.)

Eliot, many thanks for your patience. If the above makes sense I'd like to send a test covering #terminateTo along with an explanation.

PS: there's a recent note in the comments in StackInterpreter >> #commonReturn:
"We must move any frames on intervening pages above the frame linked to because these may be in use, e.g. via co-routining"
Does it suggest not all frames are marked as returned from after all, perhaps? :)

Best,
Jaromir


--

Jaromír Matas

mail at jaromir.net


From: Eliot Miranda<mailto:eliot.miranda at gmail.com>
Sent: Monday, February 20, 2023 4:51
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] Questions about FullBlock closures

Hi Jaromir,


On Feb 18, 2023, at 11:04 PM, Jaromir Matas <mail at jaromir.net> wrote:

> No, the difference is that the pc and sender of the contexts between the receiver and argument contexts are set to nil when one does terminateTo: but not when one does privSender:.

This is crystal clear, no problem here

> Clearly they *must* be do professed yo be marked as having been returned from

And this is precisely where I‘m lost: Why do the (cut out) contexts *need* to be marked as returned from? In order to be GC’d? Otherwise they’d get stuck? Looking at StackInterpreterPrimitives >> #primitiveTerminateTo there’s a lot going on with frames and pages too but when I replace terminateTo: with privSender: in my image (two/three senders only), seemingly everything works as before (tests ok etc) which drives me crazy :)

Remember we talked about blocks and non-local return? Let’s imagine that somewhere along the chain between the receiver and argument of terminateTo: a block with a up arrow return was created whose home was on that chain.  Were the contexts not marked as having been returned from then a subsequent activation of the block might actually be able to return.  This is a no no.



Thanks for your patience,
Jaromir


Eliot
_,,,^..^,,,_ (phone)




On Feb 17, 2023, at 2:45 PM, Jaromir Matas <mail at jaromir.net> wrote:

Hi Eliot, Tim, all

> Eliot: But I suspect that atomicity is not crucial and that therefore the terminateTo: primitive is only an optimization.

That explains why the *primitive* is not essential; another question is whether #terminateTo: as such (primitive or not) is essential for e.g. freeing the stack frames/pages or "just" an optimization.

I'll try to answer this question. After reading Eliot's post at http://www.mirandabanda.org/cogblog/2009/01/14/under-cover-contexts-and-the-big-frame-up/ and checking Tim's primitiveTerminateTo: code, my guess is that the difference between

thisContext terminateTo: someContext.

and

thisContext privSender: someContext.

is that in the former case the stack frames/pages are being freed immediately, while in the latter case the whole sender chain remains intact until it is, sooner or later, GCd or, if referenced, remains available for further use (whatever it may be).

No, the difference is that the pc and sender of the contexts between the receiver and argument contexts are set to nil when one does terminateTo: but not when one does privSender:.  Clearly they *must* be do professed yo be marked as having been returned from (c.f. earlier parts of the conversation).




That may be why using #terminateTo: (primitive or not) is not essential, but an optimization. Too naive? :) Could there be, perhaps, some circularities that need breaking? Or is GC too slow to clean up after non-local returns? (btw I only found two senders of #terminateTo: - #runUntilErrorOrReturnFrom: and #resume:[through:]).

Oops… more questions popping up :)

Best,
Jaromir


--

Jaromír Matas

mail at jaromir.net


From: Jaromir Matas<mailto:mail at jaromir.net>
Sent: Sunday, February 12, 2023 21:06
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] Questions about FullBlock closures

Hi Eliot, Tim,

thanks a lot for your answers.

> peek at the generated gcc3x-cointerp.c file and find the primitiveTerminateTo() routine

Awfully complex :)

> But I suspect that atomicity is not crucial and that therefore the terminateTo: primitive is only an optimization.

To be sure I understand, please let me try from another angle:
Consider a thought experiment: What happens if, in Context >> #resumeEvaluating:, we replace

                thisContext terminateTo: self.

with:

                thisContext privSender: self.

Would garbage collection/the vm take care of freeing the intervening contexts/frames or is in fact #terminateTo: prim 196 essential for this?

When I try that, nothing wrong seems to happen and all tests (Exception, Process, Methods, Object, Classes) work fine but I'm only looking at the surface of things indeed.

What I'm trying to achieve is to reuse the existing #unwindTo: method in #resumeEvaluating:, which seems to work fine, and just wonder whether replacing the #terminateTo: with #privSender: would further simplify the method (or made the system less efficient):

Context>>resumeEvaluating: aBlock
                "Unwind thisContext to self and resume with value as result of last send.
                 Execute unwind blocks when unwinding.
                ASSUMES self is a sender of thisContext"

                self isDead ifTrue: [self cannotReturn: aBlock value to: self].
                thisContext
                                unwindTo: self safely: false;
                                terminateTo: self.
                ^aBlock value

Thanks,
best,
Jaromir


--

Jaromír Matas

mail at jaromir.net


From: Eliot Miranda<mailto:eliot.miranda at gmail.com>
Sent: Sunday, February 12, 2023 16:54
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] Questions about FullBlock closures

Hi Jaromir,

On Feb 12, 2023, at 5:13 AM, Jaromir Matas <mail at jaromir.net> wrote:

Hi,

>> What is the purpose of the primitive 196 Context>>#terminateTo:?

> Strongly related to prim 195 & 197. You have to check each context up the chain to see what exceptions might be dealt with & by which context.

My understanding is prim 195 (#findNextUnwindContextUpTo:) searches for unwind contexts and prim 197 (#findNextHandlerContextStarting) searches for handler context. But prim 195 (#terminateTo:) is unclear to me. #terminateTo comment says:

"Terminate all the Contexts between me and previousContext, if previousContext is on my Context stack. Make previousContext my sender."

Which doesn't say much except it's nilling all context's pc and sender between self and the argument. I wonder what else it does behind the scenes and why :)

In a paper from 2009 Eliot wrote:
"[...] there’s a primitive terminateTo: that does something similar to the context nilling in non-local return. It is a little simpler than non-local return (although not much) but it is just a variation on the same theme so I’ll spare you."

I wish Eliot didn't spare us :) Other than that I haven't found any further info.

Because the primitive 196 dates back to 2001 I'm unable to judge whether it's still essential to nil the intermediate contexts (to free stack pages or something).

In particular I'm trying to understand whether the use of #terminateTo *primitive* in #resumeEvaluating: and #resume:through: is essential or not.

There are three things that primitive 296, Context>>terminateTo: does, or rather there are two things it does, and it does them in a particular way. It sets senders and pcs of intervening contexts to nil. It sets the sender of the receiver to the argument. If does this atomically.

I can’t say whether the last of these is necessary. If it is, then the primitive is needed, essential. But the primitive merely optimized the former two operations.

In the StackInterpreter & CoInterpreter it provides a significant optimization because

- the primitive avoids creating any contexts; if if were not implemented primitively then  contexts would have to be created for all intervening frames, merely to throw unreferenced ones away

- intervening stack pages can be freed without looking at their contents

So the primitive is an important optimization for Stack/CoInterpreter.

But I suspect that atomicity is not crucial and that therefore the terminateTo: primitive is only an optimization.

 Any additional info greatly appreciated.

HTH

Thanks,
Jaromir


--

Jaromír Matas

mail at jaromir.net


From: tim Rowledge<mailto:tim at rowledge.org>
Sent: Sunday, February 12, 2023 5:18
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] Questions about FullBlock closures



> On 2023-02-11, at 4:04 PM, Jaromir Matas <mail at jaromir.net> wrote:
>
> If I may, I have a question here: What is the purpose of the primitive 196 Context>>#terminateTo:? Why primitive and why terminating each context along the way? Naively, I'd think checking the two contexts are in the same sender chain and patching them via privSender should do but I'm sure I'm missing something here :)
>

Exception handling stuff. Strongly related to prim 195 & 197. You have to check each context up the chain to see what exceptions might be dealt with & by which context. Look at e.g. Process>>#terminate for some context.


tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
When flying inverted, remember that down is up and up is expensive






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20230222/245d3564/attachment.html>