Hi Eliot,

> Look at SystemNavigation unboundMethods [...]

SystemNavigation new allUnboundMethods

Yep, they're there! Not that I fully understand what I'm looking at but I can see where the BlockClosure instances are coming from; thanks.

I guess #unusedBlocks method is no longer working in Squeak and Pharo as push closure instruction doesn't have 4 bytes any longer (in Cuis it still works though as they don’t use FullBlockClosures):

SystemNavigation new unusedBlocks "FAIL"

> The notion is “clean blocks”. I believe Pharo has so extended their compiler.

Yes, and still experimental/disabled.

Btw, I can see they've removed all BlockClosure instances and even the supporting code! All closures are FullBlockClosures.

> I hope that having read and thought through the above you now feel you’re no longer missing something.

Yes, MANY thanks again!!

Except, as asked in the other mail, I'd really like to understand #terminateTo: primitive's purpose :)

Best,
Jaromir


--

Jaromír Matas

mail at jaromir.net


From: Jaromir Matas<mailto:mail at jaromir.net>
Sent: Sunday, February 12, 2023 1:04
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: RE: [squeak-dev] Questions about FullBlock closures

Hi Eliot,
many thanks for your thorough response.

> If a block can outlive its outer activation it can outlive the recompilation of its method.

That's what I suspected but wasn't sure, thanks.

> The notion is “clean blocks”.
>     [:a :b| a asInteger < b asInteger]

Ahh, similar idea to "pure functions" ? I've noticed #isClean method but haven't seen much usage (yet?). Thanks for the pointer.

> a “sideways return”.  This is where a block is forked into another process and the up arrow return done from within that (or some nested) block in the other process.  This is clearly a very bad thing

I know what you mean ([^2] fork); I guess there are cases when returning might make some sense, maybe. Your example demonstrates the second failing non-local return scenario where you try to return to a context already returned to before, which I guess is really fatal.

>> I wonder: if we used a FullBlockClosure with outerContext == nil, how would then methods like Context >> home work? They assume 'closureOrNil outerContext' is not nil and would fail, if I understand correctly.

> They won’t and can’t. They must answer nil or raise an error.

Context>>home
                closureOrNil ifNotNil:
                                [^closureOrNil outerContext home]

What I meant was if 'closureOrNil outerContext' is nil then sending #home will cause a walkback. I was just wondering whether a nil check was missing and should be e.g.:
Context>>home
                closureOrNil ifNotNil:
                                [^closureOrNil outerContext ifNotNil: [:outer | outer home]]

> Here’s an exercise for you. Find out where and how in the Cog/OpenSmalltalk VMMaker source the protection against sideways returns is implemented.

I know what to look for: vm needs to make sure home is on the sender chain (which is not in case of [^2] fork, where we have two separate sender chains). So, the vm walks the sender chain until it finds home (checking for unwind contexts along the way, via self findUnwindThroughContext: home). If home is not found then raise cannotReturn, if unwind contexts found then call aboutToReturn: and otherwise just return.

If I may, I have a question here: What is the purpose of the primitive 196 Context>>#terminateTo:? Why primitive and why terminating each context along the way? Naively, I'd think checking the two contexts are in the same sender chain and patching them via privSender should do but I'm sure I'm missing something here :)

Thanks again; it's a lot to process. I really appreciate your help.
best,
Jaromir


--

Jaromír Matas

mail at jaromir.net


From: Eliot Miranda<mailto:eliot.miranda at gmail.com>
Sent: Saturday, February 11, 2023 19:48
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] Questions about FullBlock closures

Hi Jaromir,

On Feb 11, 2023, at 5:09 AM, Jaromir Matas <mail at jaromir.net> wrote:

Hi,

Two questions:

1) When I do:
BlockClosure allInstances size "---> 66"
I always get 66 instances of BlockClosure, all of them probably related to the UI (as far as I can tell).
Q: Why do we have those closures as instances of BlockClosure and not FullBlockClosure? (just trying to understand)

Blocks can outlive their enclosing activation:

captureBlock
        GlobalVariable := [‘this is a value returned by a block whose enclosing activation has been returned from’].
        ^self

self captureBlock. GlobalVariable value

And GlobalVariable outerContext sender isNil

If a block can outlive its outer activation it can outlive the recompilation of its method.  Unless such global variables are reinitialized after the system is recompiled obsolete methods (and indeed transitively obsolete classes) can be retained.

Look at SystemNavigation unboundMethods and obsoleteClasses fir code that ferrets out these stale references.


2) When I do:
(FullBlockClosure allInstances select: [:each | each outerContext isNil]) size "---> 0"
I always get zero; this means there are no closures without the outer context set. However, the class comment says:
FullBlockClosure: [...] outerContext can be either a Context or nil.
Q: When would it make sense to use a FullBlockClosure with nilled outer context?

The notion is “clean blocks”. Blocks can close over the receiver, arguments, and local temporary variables if the methods (& blocks) in which they are nesting.  Further, a block may contain an up arrow (method) return (*). These returns attempt to return to the sender of the home context (the context for the method activation). And if course a block can contain both an up arrow return and close over self & outer arguments/variables. These three kinds of blocks are not clean.

Blocks which make no such references to their outer context are clean.  Clean blocks refer only to literals and their own arguments and local temporaries.  Such blocks are seen used as, for example, sort blocks:
    [:a :b| a asInteger < b asInteger]

In this case there is no need for the outerContext to exist and the compiler could be upgraded to instantiate them at compile time, included as literals in the literal frame.  I believe Pharo has so extended their compiler.  Note also that this is independent of the bytecode set.  (I’m not 100% sure but at least in theory) Clean FullBlockClosures could be created for V3PlusClosures methods since the value primitives are not specific to the bytecode set.

(*) rewrite the above example as

captureBlock
        GlobalVariable := [ ^ ‘this is a value returned by a block whose enclosing activation has been returned from’].
        ^self

self captureBlock. GlobalVariable value

and we now get a CannotReturnError.

In blue book Smalltalk-80 it is possible to do a “sideways return”.  This is where a block is forked into another process and the up arrow return done from within that (or some nested) block in the other process.  This is clearly a very bad thing (tm), and is explicitly banned in all modern Smalltalk implementations, Cog included.

No such situation in the system?

Not yet. We would have to modify the compiler.  The performance gains are small in general but large in specific cases.  The only real downside is that, eg in the debugger, one cannot tell in what context the clean block was created.  This may be confusing, analogously to your curiosity about the origins of the non-full blocks persisting in the system.


I wonder: if we used a FullBlockClosure with outerContext == nil, how would then methods like Context >> home work? They assume 'closureOrNil outerContext' is not nil and would fail, if I understand correctly.

They won’t and can’t. They must answer nil or raise an error.

Am I missing something/confused??

You’re asking important questions that every Smalltalk vm and/or bytecode compiler programmer should understand.  This stuff used to be discussed/taught up front.  I hope that having read and thought through the above you now feel you’re no longer missing something.

 Many thanks for any help,

Here’s an exercise for you. Find out where and how in the Cog/OpenSmalltalk VMMaker source the protection against sideways returns is implemented.

 best,
Jaromir

Cheers!
Eliot
_,,,^..^,,,_ (phone)



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20230212/9aaf2ae0/attachment.html>