Hi Nicolas,

From: Nicolas Cellier<mailto:nicolas.cellier.aka.nice at gmail.com>
Sent: Friday, May 13, 2022 19:05

Hi all,

Le ven. 13 mai 2022 à 00:47, Jaromir Matas <mail at jaromir.net<mailto:mail at jaromir.net>> a écrit :
Hi Eliot, Nicolas, all

I suspect some of my questions might have been interpreted other than intended.

>> (me) I'm so confused...

> (Eliot) Why?  We're all flailing about here :-)  It's OK for you to say to Nicolas "I think that's too complicated".

>> (me) Because once #suspend in a newer image running on an older VM falls back to the old prim 88 (old semantics), the tests and the few methods depending on the particular suspend semantics won't work correctly. Or is this the intended behavior?

> (Eliot) Of course no one intends things not to work correctly.  You have a good understanding of what's going on here, as good as, or better than, anybody else.  So you can be authoritative.

Oh dear, I hope it didn't sound ironic! The meaning was: Is the intended behavior to fall back to the old prim 88 to allow newer images to run on older VMs? Because there'd be a problem with a few tests and methods that wouldn't work... Nicolas's proposal made a perfect sense to me until Eliot's backwards compatibility explanation: I haven't even realized I was attempting the backwards compatibility the other way round :)

Yes I have proposed to fallback to primitive 88 for being very conservative and cautious.
Note that I have several VM, some from before the HighDPI changes, so maybe I'm biased.
I suspected that the loss of capacity to suspend would be catastrophic.
Another reason why I proposed that way is because I also see that the old suspend had a fallback code.
So I adopted the style of being not less cautious than predecessors.

Yes, same here, originally I even thought there would be a choice for the user to choose a suspend primitive based on her application and thus terminate, suspend etc. would have to work for all combinations :) But Eliot’s KISS approach would mean a great simplification indeed – however a decision which approach it is requires deep knowledge of what the real needs of the existing (and potential) users are – and I have zero of it :) I very much look forward to learning what the final decision will be – KISS or conservative.

KISS would only replace the primitive number in #suspend and leave the fallback code in place. Your conservative approach would mean to put two alternative code fragments into those few tests and methods (signalException, signalWaitingProcess etc) depending on the two suspend semantics – doable, not even too much work but would make things way more intertwined and more difficult to maintain in the future.

As for your level of authority, it's certainly not ironic.

Just to make sure: I hoped my comment " Or is this the intended behavior?" didn’t sound ironic ;)

You have put deep thoughts and efforts in cleaning up those dusty and brainstorming parts of the system.
If those changes are not integrated yet, it's indeed because there is no greater authority to do so!
I only try to help so that we don't waste those efforts.

Thank you!

> (Eliot) KISS is something (as a stupid person) I live by ;-)

Indeed, it would be a relief for me to keep it simpler, without the need to support newer images running on older VMs :) And without (my) even more convoluted comments :)

Eliot, thanks a lot for encouragement! My Smalltalk gut feeling is still very underdeveloped hence the confusion ;)

Apologies to Nicolas and all for any misunderstandings.
Best,
Jaromir

Jaromir, no need to apologize, I see no offense in contradiction.
My presumed level of authority should never be intimidating nor an obstacle to contradiction.
And my true level of authority should be downgraded - proportionally to the level of investment I can put in Squeak currently ;)


--

Jaromír Matas

mail at jaromir.net<mailto:mail at jaromir.net>

From: Eliot Miranda<mailto:eliot.miranda at gmail.com>
Sent: Thursday, May 12, 2022 21:24
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] SIGTRAP when Proceeding from BlockCannotReturn



On Wed, May 11, 2022 at 12:59 PM Jaromir Matas <mail at jaromir.net<mailto:mail at jaromir.net>> wrote:
Hi Eliot, Nicolas,

Thanks for your explanation in Vm-dev list (I was not sure where to put my follow-up). If I understand correctly, we don't want to put an unnecessary effort into keeping new images compatible with older VMs:

> (Eliot) [...] But a new image may expect a feature added to a contemporary VM that is absent in previous versions of the VM.
>
> This is analogous to, for example, binaries compiled against newer instructions added to the x86 such as mmx, sse 1, 2 & 3. We expect that the new processor can run older binaries; we do not expect new binaries to run on older processors unless we compile them specially.

But in that case Nicolas's proposed #suspend in this thread would do exactly that - make newer images compatible with the older VMs (with prim 88 only):

I don't see the point of doing that.  It just makes things more confusing.  KISS is something (as a stupid person) I live by ;-)

> (Nicolas) [...] Yes, I propose the selector
> Process>>suspendAndUnblock <primitive: 88> snip...
> Process>>suspend would call the new primitive, and fallback to suspendAndUnblock in case of failure (for example if new primitive is absent).

I'm so confused...

Why?  We're all flailing about here :-)  It's OK for you to say to Nicolas "I think that's too complicated".
Because once #suspend in a newer image running on an older VM falls back to the old prim 88 (old semantics), the tests and the few methods depending on the particular suspend semantics won't work correctly. Or is this the intended behavior?

Of course no one intends things not to work correctly.  You have a good understanding of what's going on here, as good as, or better than, anybody else.  So you can be authoritative.

Please advise :)

Jaromir


--

Jaromír Matas

mail at jaromir.net<mailto:mail at jaromir.net>

From: Jaromir Matas<mailto:mail at jaromir.net>
Sent: Wednesday, May 11, 2022 14:49
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] SIGTRAP when Proceeding from BlockCannotReturn


Hi Eliot,

> > On Tue, May 10, 2022 at 1:36 PM Jaromir Matas <mail at jaromir.net<mailto:mail at jaromir.net>> wrote:
> > Hi Eliot,
> >
> > Thanks a lot for your answers.
> >
> > One last question: what about Process >> #signalException:? You wrote a fix (enclosed) – is it ok to merge it when the #suspend method is updated (Nicolas’s way)?
> >
>
> Yes, I think so.  We have to test it but it seems right.  And we can go even simpler, just have the first clause as guarded by processSuspensionUnblocks ifFalse:, once we're ready to release on VMs with the new primitive.
>

Thanks again; this brings me to another question: if you go simpler, i.e. with the ifFalse clause only, it would mean the method wouldn't work correctly with older VMs with the older prim 88 suspend semantics. I was under the impression to keep compatibility with older VMs but maybe it's a nonsensical assumption on my part (The reason I'm asking is the #terminate I wrote is compatible with both the new and old VMs - but could be simplified a bit if the old VMs were not supposed to be used with the new images).

Thanks,

Jaromir


--

Jaromír Matas

mail at jaromir.net<mailto:mail at jaromir.net>

From: Eliot Miranda<mailto:eliot.miranda at gmail.com>
Sent: Tuesday, May 10, 2022 23:21
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] SIGTRAP when Proceeding from BlockCannotReturn



On Tue, May 10, 2022 at 1:36 PM Jaromir Matas <mail at jaromir.net<mailto:mail at jaromir.net>> wrote:
Hi Eliot,

Thanks a lot for your answers.

One last question: what about Process >> #signalException:? You wrote a fix (enclosed) – is it ok to merge it when the #suspend method is updated (Nicolas’s way)?

Yes, I think so.  We have to test it but it seems right.  And we can go even simpler, just have the first clause as guarded by processSuspensionUnblocks ifFalse:, once we're ready to release on VMs with the new primitive.


Thanks again,
Jaromir


--

Jaromír Matas

+420 777 492 777
mail at jaromir.net<mailto:mail at jaromir.net>

From: Eliot Miranda<mailto:eliot.miranda at gmail.com>
Sent: Tuesday, May 10, 2022 21:17
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] SIGTRAP when Proceeding from BlockCannotReturn

Hi Jaromir,

On Tue, May 10, 2022 at 8:31 AM Jaromir Matas <mail at jaromir.net<mailto:mail at jaromir.net>> wrote:
Hi Eliot, Nicolas,

> > > (Eliot) Apologies for not responding sooner.  It seems to me the semantics must remain as they are, and it be left up to the image to bind suspend to the primitive most convenient. Then backwards compatibility can be provided for cases such as DelayWaitTimeout by providing a renamed suspend which accesses the old primitive.
> >
> > (Nicolas) Yes, I propose the selector
> Process>>suspendAndUnblock <primitive: 78> snip...
> Process>>suspend would call the new primitive, and fallback to suspendAndUnblock in case of failure (for example if new primitive is absent).
> The question remaining is if we really need two new primitives 578 (V1) and 588 (V2).
> It seems like signalException: would require V1, while Jaromir terminate logic would require V2...
>
> (Eliot) +1. Works for me.  Having both 578 & 588 available in the VM is fine. I agree that surfacing only one of them is best.

Thank you, I wasn't sure how to proceed with this :) Do I understand correctly #suspend will have the new suspend semantics (either V1 or preferably V2) in new images (new release) and the previous semantics (via prim 88) will be available for backward compatibility (or special usage)?

Yes, exactly.
In that case the tests I submitted in KernelTests-jar.421 should work without changes...

Cool!

The next question is what to do with:
Process >> suspendPrimitivelyOrFail
                "Test support. Execute primitive 88, or fail."

                <primitive: 88>
                ^self primitiveFailed

Change it or remove it? Replace with Nicolas’s Process>>suspendAndUnblock ?

Well it appears to be used only in testAtomicSuspend.  So I would change it to use the same primitive number as suspend, which I guess is 578, primitiveSuspendBackingUpV2, which has the more consistent semantics.


Thanks!

Best,

Jaromir

From: Eliot Miranda<mailto:eliot.miranda at gmail.com>
Sent: Monday, May 9, 2022 23:23
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] SIGTRAP when Proceeding from BlockCannotReturn



On May 9, 2022, at 1:04 PM, Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com<mailto:nicolas.cellier.aka.nice at gmail.com>> wrote:


Hi Eliot,
Le lun. 9 mai 2022 à 20:36, Eliot Miranda <eliot.miranda at gmail.com<mailto:eliot.miranda at gmail.com>> a écrit :
Hi Jaromir, Hi Nicolas,

On Sun, May 8, 2022 at 8:17 AM Jaromir Matas <mail at jaromir.net<mailto:mail at jaromir.net>> wrote:
Hi Nicolas,

>> About the latest VM, (Smalltalk processSuspensionUnblocks) answers about the behavior of primitiveSuspend (#88).
> Err,  wrong wording (Smalltalk processSuspensionUnblocks) does not describe the behavior of primitiveSuspend #(88).
> It answers whether the VM has primitives #578 and #588 (see #getCogVMFeatureFlags in VMMaker)...
>
Yes, thanks, well that was the original idea for Smalltalk processSuspensionUnblocks  to answer about the behavior of the primitiveSuspend #88; which was before the primitives #568 and #578 were introduced. (And which was when I wrote the tests using this original logic). However that approach ran into problems with some existing implementations (e.g. Virtend) and as a result the two new primitives were introduced (#568 and #578) - and apparently the logic of what Smalltalk processSuspensionUnblocks answers about changed. Since then I haven't heard from Eliot about his plans how to proceed with the new suspend semantics or whether this is it :)

Apologies for not responding sooner.  It seems to me the semantics must remain as they are, and it be left up to the image to bind suspend to the primitive most convenient. Then backwards compatibility can be provided for cases such as DelayWaitTimeout by providing a renamed suspend which accesses the old primitive.
Yes, I propose the selector
Process>>suspendAndUnblock <primitive: 78> snip...
Process>>suspend would call the new primitive, and fallback to suspendAndUnblock in case of failure (for example if new primitive is absent).
The question remaining is if we really need two new primitives 578 (V1) and 588 (V2).
It seems like signalException: would require V1, while Jaromir terminate logic would require V2...

+1. Works for me.  Having both 578 & 588 available in the VM is fine. I agree that surfacing only one of them is best.


Sorry about not renaming processSuspensionUnblocks.  This should be something like Smalltalk hasExtendedProcessSuspensionSemantics or some such.

It's not too late, there are not many usages, except inbox experimentations.




In case this is still WIP I'd quite like an approach similar to Smalltalk processPreemptionYields - you'd set Smalltalk processSuspensionUnblocks true or false depending on how you want the VM to behave and the VM would use the right semantics (unless this is too naive :) ).

Regarding the tests failing with the newest VM:
testAtomicSuspend - we can remove this updated version and leave the existing one for the moment
testRevisedSuspendExpectations - we can leave this one out too
testTerminateBlockedInNestedEnsure1 - I'll take a look at these two and try to adjust the logic
testTerminateBlockedInNestedEnsure2

All remaining test in KernelTests-jar.421 work as intended.

Thanks again,
Jaromir


--

Jaromír Matas

mail at jaromir.net<mailto:mail at jaromir.net>

From: Nicolas Cellier<mailto:nicolas.cellier.aka.nice at gmail.com>
Sent: Sunday, May 8, 2022 16:22
To: The general-purpose Squeak developers list<mailto:squeak-dev at lists.squeakfoundation.org>
Subject: Re: [squeak-dev] SIGTRAP when Proceeding from BlockCannotReturn



Le dim. 8 mai 2022 à 15:59, Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com<mailto:nicolas.cellier.aka.nice at gmail.com>> a écrit :


Le dim. 8 mai 2022 à 14:16, Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com<mailto:nicolas.cellier.aka.nice at gmail.com>> a écrit :
Hi Jaromir

Le sam. 7 mai 2022 à 14:45, Jaromir Matas <mail at jaromir.net<mailto:mail at jaromir.net>> a écrit :
Hi Marcel, Nicolas, Eliot,


> (Nicolas) I have merged Kernel-jar.1446 which is trivial (re-signal an Exception).

Thanks!

> (Marcel) Hmm... #testSimpleResignalVsOuter1 is still failing.

In my latest trunk image it passes ok (5/7/22)

>> (Marcel) I will also take a look at KernelTests-jar.421 to check whether any new semantics are okay.
>>
> (Nicolas) Yes, I did that and got two failing tests
> testTerminateTerminatingProcess
> testResumeTerminatingProcess
>
> both failures look the same, in second self assert: terminator isSuspended.

Yes, they are supposed to fail at the moment so I suggest to make them expected failures/feature requests. However there's a stupid bug in both I’ve fixed now: they were indeed supposed to fail the following assertion:
``` self should: [terminatee terminate] raise: Error. ```

Apologies for the confusion - an updated changeset is enclosed.


However, there's another issue regarding the revised #suspend semantics Eliot has been working on. I've updated the process tests in KernelTests-jar.421 to test both the old and the new #suspend semantics. The two semantics should be distinguishable via Smalltalk processSuspensionUnblocks flag answering true in case of the old semantics and false in case of the revised one; my updated tests use this logic. However, unfortunately the latest VM answers "false" but uses the OLD suspend semantics in #suspend prim 88.

So I'm surprised you haven't observed more tests failing due to the new suspend semantics... What VM have you used – an older one? I'm on the latest trunk's VM 3183. And what is the answer of Smalltalk processSuspensionUnblocks? :) I'm utterly confused...

Hi Eliot - I may be confused here but if the current VM uses the old #suspend prim 88 semantics, shouldn't ```Smalltalk processSuspensionUnblocks``` answer true?


Indeed, I can confirm that the latest VM does answer false to (Smalltalk processSuspensionUnblocks).
Though, process suspension still unblocks as shown by the example at
https://github.com/pharo-project/pharo/issues/10669

A bit more simply, this answers false - it shouldn't:

s := Semaphore new.
p := [s wait] newProcess.
p resume.
100 milliSeconds wait.
p suspend; resume.
100 milliSeconds wait.
p isTerminated = Smalltalk processSuspensionUnblocks.

technically, this answers false - it shouldn't:

s := Semaphore new.
p := [s wait] newProcess.
p resume.
100 milliSeconds wait.
p suspend == s = Smalltalk processSuspensionUnblocks


About the latest VM, (Smalltalk processSuspensionUnblocks) answers about the behavior of primitiveSuspend (#88).
Err,  wrong wording (Smalltalk processSuspensionUnblocks) does not describe the behavior of primitiveSuspend #(88).
It answers whether the VM has primitives #578 and #588 (see #getCogVMFeatureFlags in VMMaker)...

The new Behavior is implemented by primitiveSuspendBackingUpV1 (#578) and primitiveSuspendBackingUpV2 (#588).
V1 always answers the old list (even if a Semaphore/Mutex), V2 answers nil in case of Semaphore/Mutex.

So, only ((Process>>#suspend) primitive) can give you a clue about the behavior, assuming that the primitive in question is implemented...

So it depends if we want to make it a Preference, or make the image compatible with some less capable VM...

we could implement
    Process>>primitiveSuspendAndUnblock as <primitive: 88>
    Process>>primitiveSuspend as <primitive: 588>
test the VM flag in suspend
    Process>>suspend
        ^(VMHasPrimitiveSuspendBackingUp ifNil: [VMHasPrimitiveSuspendBackingUp := Smalltalk processSuspensionUnblocks not])
            ifTrue: [self primitiveSuspend]
            ifFalse: [self primitiveSuspendAndUnblock]
Then arrange to reset VMHasPrimitiveSuspendBackingUp to nil at snapshot...

I will tell next week about the status of the VM which I used for testing, it's possibly a few months old.

> (Nicolas) I would also consider 1445 1421 and most importantly 1447.

Regarding 1421: alternatively, you might consider a newer version 1415 where the only difference is #return:from: passes a block instead of nil to #resume:through: to cause a fresh search for the first unwind context; otherwise they are equivalent.

OK, I will check again.

> (Marcel) Maybe we can just merge
>
> KernelTests-jar.393
> KernelTests-jar.418
> KernelTests-jar.421
>
> And go from there? Or are there any objections?

KernelTests-jar.393 is just a special case of a more general KernelTests-jar.418 test…

In my image all tests are green provided Smalltalk processSuspensionUnblocks answers true and the two above tests Nicolas mentioned are marked expected failures :)

Thanks very much for your feedback,
Jaromir


--

Jaromír Matas

mail at jaromir.net<mailto:mail at jaromir.net>





--
_,,,^..^,,,_
best, Eliot






--
_,,,^..^,,,_
best, Eliot




--
_,,,^..^,,,_
best, Eliot





--
_,,,^..^,,,_
best, Eliot



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20220513/28183ef2/attachment-0001.html>