On 2021-01-22, at 5:26 PM, David T. Lewis lewis@mail.msen.com wrote:

On Fri, Jan 22, 2021 at 02:03:10PM -0600, jaromir wrote:

...

I ran the following in Squeak 5.3:

TranscriptStream forceUpdate: false. [ 10000 timesRepeat: [ Transcript show: 'x' ] ] forkAt: 39

When the Transcript window fills up, 'Message not understood' and 'Assertion failure' appear. I'm wondering is this a bug? Thanks. Jaromir

http://forum.world.st/file/t372955/Screenshot_2021-01-22_205244.png

It certainly does appear to be a bug.

We had some discussion about it not so long ago; you can't use the Transcript to log bugs from Seaside very well, for example. Background processes, mutexs, that sort of thing. IIRC Levente offered an explanation?

tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim The hardness of the butter is proportional to the softness of the bread.

It's simply not thread-safe.

Hmm... in particular, there is a side effect (i.e. layout update) in the window's pluggable text morph. Even if #forceUpdate is false.

In #endEntry, we could check for "Processor activeProcess == Project current uiProcess" to then use "Project current addDeferredUIMessage..." automatically.

Best, Marcel Am 23.01.2021 21:05:05 schrieb Levente Uzonyi leves@caesar.elte.hu: On Fri, 22 Jan 2021, tim Rowledge wrote:

...

On 2021-01-22, at 5:26 PM, David T. Lewis wrote:

On Fri, Jan 22, 2021 at 02:03:10PM -0600, jaromir wrote:

...

I ran the following in Squeak 5.3:

TranscriptStream forceUpdate: false. [ 10000 timesRepeat: [ Transcript show: 'x' ] ] forkAt: 39

When the Transcript window fills up, 'Message not understood' and 'Assertion failure' appear. I'm wondering is this a bug? Thanks. Jaromir

It certainly does appear to be a bug.

We had some discussion about it not so long ago; you can't use the Transcript to log bugs from Seaside very well, for example. Background processes, mutexs, that sort of thing. IIRC Levente offered an explanation?

It's simply not thread-safe. If you want it to be thread-safe, you can pass your message to the UI process to show it. Make sure your message is computed in its own process and is passed as a precomputed string to avoid other kinds of race conditions. E.g.:

| theMessage | theMessage := 'Something very {1}.' format: { 'important' }. Project current addDeferredUIMessage: [ Transcript show: theMessage; cr ].

Levente

tim

tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim The hardness of the butter is proportional to the softness of the bread.

But wouldn't violate that the #forceUpdate preference when a non-UI process blocks the image for a while?

... "violate" is such a strong word. ^__^ I know that we have a "lock" in Transcript to at least not mix up the buffer contents. "forceUpdate" could be enabled by adding an extra semaphore to let the non-ui process sleep until that deferred UI message got processed.

Best, Marcel Am 25.01.2021 13:24:51 schrieb Thiede, Christoph christoph.thiede@student.hpi.uni-potsdam.de:

In #endEntry, we could check for "Processor activeProcess == Project current uiProcess" to then use "Project current addDeferredUIMessage..." automatically.

But wouldn't violate that the #forceUpdate preference when a non-UI process blocks the image for a while?

Best, Christoph Von: Squeak-dev squeak-dev-bounces@lists.squeakfoundation.org im Auftrag von Taeumel, Marcel Gesendet: Montag, 25. Januar 2021 12:25:56 An: squeak-dev Betreff: Re: [squeak-dev] Transcript error when forceUpdate: false (?)

It's simply not thread-safe.

Hmm... in particular, there is a side effect (i.e. layout update) in the window's pluggable text morph. Even if #forceUpdate is false.

In #endEntry, we could check for "Processor activeProcess == Project current uiProcess" to then use "Project current addDeferredUIMessage..." automatically.

Best, Marcel Am 23.01.2021 21:05:05 schrieb Levente Uzonyi leves@caesar.elte.hu: On Fri, 22 Jan 2021, tim Rowledge wrote:

...

On 2021-01-22, at 5:26 PM, David T. Lewis wrote:

On Fri, Jan 22, 2021 at 02:03:10PM -0600, jaromir wrote:

...

I ran the following in Squeak 5.3:

TranscriptStream forceUpdate: false. [ 10000 timesRepeat: [ Transcript show: 'x' ] ] forkAt: 39

When the Transcript window fills up, 'Message not understood' and 'Assertion failure' appear. I'm wondering is this a bug? Thanks. Jaromir

It certainly does appear to be a bug.

We had some discussion about it not so long ago; you can't use the Transcript to log bugs from Seaside very well, for example. Background processes, mutexs, that sort of thing. IIRC Levente offered an explanation?

It's simply not thread-safe. If you want it to be thread-safe, you can pass your message to the UI process to show it. Make sure your message is computed in its own process and is passed as a precomputed string to avoid other kinds of race conditions. E.g.:

| theMessage | theMessage := 'Something very {1}.' format: { 'important' }. Project current addDeferredUIMessage: [ Transcript show: theMessage; cr ].

Levente

tim

tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim The hardness of the butter is proportional to the softness of the bread.

Well, I tried deferring the whole Transcript endEntry machinery to the UI doOneCycle (bypassing the changed: #appendEntryLater mechanism) for #forceUpdate = false only ... and it seems to avoid the problem!

TranscriptStream >> endEntry

deferredEntry ifNil: [ false ]. "this is a new instance variable" self semaphore critical:[ self class forceUpdate ifTrue: [self changed: #appendEntry; reset] ifFalse: [deferredEntry := true].

TranscriptStream >> flushDeferredEntry "This is run every UI cycle in doOneCycleNowFor:"

deferredEntry ifTrue: [ self class forceUpdate: true. self endEntry. deferredEntry := false. self class forceUpdate: false. ]

doOneCycleNowFor: aWorld "... the whole body remains unchanged except:" capturingGesture ifFalse: [aWorld runStepMethods. Transcript flushDeferredEntry. "this is printing for #forceUpdate = false" self displayWorldSafely: aWorld].

For #forceUpdate = true the endEntry mechanism remains unchanged and failing as before...

-- Sent from: http://forum.world.st/Squeak-Dev-f45488.html

Hi Marcel,

On Tue, 26 Jan 2021, Marcel Taeumel wrote:

Hi Jaromir, please take a look at Collections-mt.923 (inbox). Maybe those changes would satisfy all your current needs in this regard. :-)

@all: Would Transcript now be thread-safe? Did I miss something?

Not at all. The problem is not #endEntry but all the other methods writing to the stream e.g.: #nextPutAll:. The real solution is to replace the stream-like API with one that provides an easy way to write messages in a thread-safe way. E.g.:

Transcript showStream: [ :stream | stream nextPutAll: 'Hello'; space; nextPutAll: ' World!'; cr ]

Which guarantees that 'Hello World!' appears on the Transcript without being mixed with other messages.

or

Transcript show: 'Hello {1}!{2}' format: { 'World'. String cr }.

The implementations could be e.g.:

showStream: aBlock

| string | string := String streamContents: aBlock. self safelyNextPutAll: string

show: aString format: arguments

| string | string := aString format: arguments. self safelyNextPutAll: string

The last challenge is to implement #safelyNextPutAll: which involves making Transcript not be a TranscriptStream. Transcript should encapsulate the stream and use a Mutex (not a global one because its pointless) to provide thread-safety while writing the characters on it or reading its contents. E.g.:

safelyNextPutAll: aString

mutex critical: [ stream nextPutAll: aString ]

For backwards compatibility, the stream-API methods must be provided for a while but those methods should be thread-safe on their own. E.g.:

nextPutAll: aString

self safelyNextPutAll: aString

nextPut: aCharacter

self safelyNextPutAll: aCharacter asString

Levente

Best, Marcel

  Am 25.01.2021 21:38:36 schrieb jaromir <m@jaromir.net>:

  Well, I tried deferring the whole Transcript endEntry machinery to the UI
  doOneCycle (bypassing the changed: #appendEntryLater mechanism) for
  #forceUpdate = false only ... and it seems to avoid the problem!

  TranscriptStream >> endEntry

  deferredEntry ifNil: [ false ]. "this is a new instance variable"
  self semaphore critical:[
  self class forceUpdate
  ifTrue: [self changed: #appendEntry; reset]
  ifFalse: [deferredEntry := true].


  TranscriptStream >> flushDeferredEntry
  "This is run every UI cycle in doOneCycleNowFor:"

  deferredEntry ifTrue: [
  self class forceUpdate: true.
  self endEntry.
  deferredEntry := false.
  self class forceUpdate: false.
  ]

  doOneCycleNowFor: aWorld

  "... the whole body remains unchanged except:"

  capturingGesture ifFalse:
  [aWorld runStepMethods.
  Transcript flushDeferredEntry. "this is printing for #forceUpdate =
  false"
  self displayWorldSafely: aWorld].


  For #forceUpdate = true the endEntry mechanism remains unchanged and failing
  as before...



  --
  Sent from: http://forum.world.st/Squeak-Dev-f45488.html

Hi

apart from the debuggers, why bother making Transcript thread-safe in the first place?

The Transcript is a mere output stream for debugging/logging etc., our stdout or stderr.

A separate kind of log (that optionally could output to the transcript) would be more apt IMHO. Thread safety could be addressed there without complicating Transcript more…

my 2ct -tobias

On 28. Jan 2021, at 09:05, Marcel Taeumel marcel.taeumel@hpi.de wrote:

Hi Levente.

Thanks für clarification. We are talking about more than one problem here. :-) I did just address the issue with errors/debuggers happening due to UI process disruptions. You are also concerned about the order of contents in the Transcript's contents. Both are very important and required for claiming that it would be thread-safe.

(Which reminds me that I find it very strange that the UI components (e.g. PluggableTextMorph) do also check the Transcript's internal characterLimit. I would expect an append-only behavior when a Transcript window is open.)

Anyway, your proposal about a safelyNextPut(All): looks good. :-) +1

Best, Marcel

...

Am 27.01.2021 23:05:21 schrieb Levente Uzonyi leves@caesar.elte.hu:

Hi Marcel,

On Tue, 26 Jan 2021, Marcel Taeumel wrote:

...

Hi Jaromir, please take a look at Collections-mt.923 (inbox). Maybe those changes would satisfy all your current needs in this regard. :-)

@all: Would Transcript now be thread-safe? Did I miss something?

Not at all. The problem is not #endEntry but all the other methods writing to the stream e.g.: #nextPutAll:. The real solution is to replace the stream-like API with one that provides an easy way to write messages in a thread-safe way. E.g.:

Transcript showStream: [ :stream | stream nextPutAll: 'Hello'; space; nextPutAll: ' World!'; cr ]

Which guarantees that 'Hello World!' appears on the Transcript without being mixed with other messages.

or

Transcript show: 'Hello {1}!{2}' format: { 'World'. String cr }.

The implementations could be e.g.:

showStream: aBlock

| string | string := String streamContents: aBlock. self safelyNextPutAll: string

show: aString format: arguments

| string | string := aString format: arguments. self safelyNextPutAll: string

The last challenge is to implement #safelyNextPutAll: which involves making Transcript not be a TranscriptStream. Transcript should encapsulate the stream and use a Mutex (not a global one because its pointless) to provide thread-safety while writing the characters on it or reading its contents. E.g.:

safelyNextPutAll: aString

mutex critical: [ stream nextPutAll: aString ]

For backwards compatibility, the stream-API methods must be provided for a while but those methods should be thread-safe on their own. E.g.:

nextPutAll: aString

self safelyNextPutAll: aString

nextPut: aCharacter

self safelyNextPutAll: aCharacter asString

Levente

...

Best, Marcel

Am 25.01.2021 21:38:36 schrieb jaromir :

Well, I tried deferring the whole Transcript endEntry machinery to the UI doOneCycle (bypassing the changed: #appendEntryLater mechanism) for #forceUpdate = false only ... and it seems to avoid the problem!

TranscriptStream >> endEntry

deferredEntry ifNil: [ false ]. "this is a new instance variable" self semaphore critical:[ self class forceUpdate ifTrue: [self changed: #appendEntry; reset] ifFalse: [deferredEntry := true].

TranscriptStream >> flushDeferredEntry "This is run every UI cycle in doOneCycleNowFor:"

deferredEntry ifTrue: [ self class forceUpdate: true. self endEntry. deferredEntry := false. self class forceUpdate: false. ]

doOneCycleNowFor: aWorld

"... the whole body remains unchanged except:"

capturingGesture ifFalse: [aWorld runStepMethods. Transcript flushDeferredEntry. "this is printing for #forceUpdate = false" self displayWorldSafely: aWorld].

For #forceUpdate = true the endEntry mechanism remains unchanged and failing as before...

-- Sent from: http://forum.world.st/Squeak-Dev-f45488.html

On 28. Jan 2021, at 10:41, Marcel Taeumel marcel.taeumel@hpi.de wrote:

Hi Tobias

...

A separate kind of log (that optionally could output to the transcript)

Transcript can use stdout already. Not sure we want to shift the problem to the user again ... :-/ "Every time you provide an option, you expect users to make a decision." :-)

That's not what I meant.

Transcript is Smalltalk for stdout, I would say. Complicating it to act like a proper logger is IMHO ill-directed.

Neither C printf nor Java System.out.println[1] nor (i think) Racket display are thread safe.

And I think this is ok.

Transcript shouldn't break when used concurrently, but I don't think it should do something sophisticated…

Transcript is not a good logger. We shouldn't pretend it is.

Maybe GemStone's object log is a neat abstraction. Thread-safe, object based, with time stamps.

Best regards -Tobias

[1]: https://stackoverflow.com/questions/32880557/is-system-out-println-thread-sa...

Best, Marcel

...

Am 28.01.2021 10:32:27 schrieb Tobias Pape das.linux@gmx.de:

Hi

apart from the debuggers, why bother making Transcript thread-safe in the first place?

The Transcript is a mere output stream for debugging/logging etc., our stdout or stderr.

A separate kind of log (that optionally could output to the transcript) would be more apt IMHO. Thread safety could be addressed there without complicating Transcript more…

my 2ct -tobias

...

On 28. Jan 2021, at 09:05, Marcel Taeumel wrote:

Hi Levente.

Thanks für clarification. We are talking about more than one problem here. :-) I did just address the issue with errors/debuggers happening due to UI process disruptions. You are also concerned about the order of contents in the Transcript's contents. Both are very important and required for claiming that it would be thread-safe.

(Which reminds me that I find it very strange that the UI components (e.g. PluggableTextMorph) do also check the Transcript's internal characterLimit. I would expect an append-only behavior when a Transcript window is open.)

Anyway, your proposal about a safelyNextPut(All): looks good. :-) +1

Best, Marcel

...

Am 27.01.2021 23:05:21 schrieb Levente Uzonyi :

Hi Marcel,

On Tue, 26 Jan 2021, Marcel Taeumel wrote:

...

Hi Jaromir, please take a look at Collections-mt.923 (inbox). Maybe those changes would satisfy all your current needs in this regard. :-)

@all: Would Transcript now be thread-safe? Did I miss something?

Not at all. The problem is not #endEntry but all the other methods writing to the stream e.g.: #nextPutAll:. The real solution is to replace the stream-like API with one that provides an easy way to write messages in a thread-safe way. E.g.:

Transcript showStream: [ :stream | stream nextPutAll: 'Hello'; space; nextPutAll: ' World!'; cr ]

Which guarantees that 'Hello World!' appears on the Transcript without being mixed with other messages.

or

Transcript show: 'Hello {1}!{2}' format: { 'World'. String cr }.

The implementations could be e.g.:

showStream: aBlock

| string | string := String streamContents: aBlock. self safelyNextPutAll: string

show: aString format: arguments

| string | string := aString format: arguments. self safelyNextPutAll: string

The last challenge is to implement #safelyNextPutAll: which involves making Transcript not be a TranscriptStream. Transcript should encapsulate the stream and use a Mutex (not a global one because its pointless) to provide thread-safety while writing the characters on it or reading its contents. E.g.:

safelyNextPutAll: aString

mutex critical: [ stream nextPutAll: aString ]

For backwards compatibility, the stream-API methods must be provided for a while but those methods should be thread-safe on their own. E.g.:

nextPutAll: aString

self safelyNextPutAll: aString

nextPut: aCharacter

self safelyNextPutAll: aCharacter asString

Levente

...

Best, Marcel

Am 25.01.2021 21:38:36 schrieb jaromir :

Well, I tried deferring the whole Transcript endEntry machinery to the UI doOneCycle (bypassing the changed: #appendEntryLater mechanism) for #forceUpdate = false only ... and it seems to avoid the problem!

TranscriptStream >> endEntry

deferredEntry ifNil: [ false ]. "this is a new instance variable" self semaphore critical:[ self class forceUpdate ifTrue: [self changed: #appendEntry; reset] ifFalse: [deferredEntry := true].

TranscriptStream >> flushDeferredEntry "This is run every UI cycle in doOneCycleNowFor:"

deferredEntry ifTrue: [ self class forceUpdate: true. self endEntry. deferredEntry := false. self class forceUpdate: false. ]

doOneCycleNowFor: aWorld

"... the whole body remains unchanged except:"

capturingGesture ifFalse: [aWorld runStepMethods. Transcript flushDeferredEntry. "this is printing for #forceUpdate = false" self displayWorldSafely: aWorld].

For #forceUpdate = true the endEntry mechanism remains unchanged and failing as before...

-- Sent from: http://forum.world.st/Squeak-Dev-f45488.html

Hi

On 28. Jan 2021, at 13:31, Levente Uzonyi leves@caesar.elte.hu wrote:

Hi Tobias,

On Thu, 28 Jan 2021, Tobias Pape wrote:

...

...

On 28. Jan 2021, at 10:41, Marcel Taeumel marcel.taeumel@hpi.de wrote: Hi Tobias

...

A separate kind of log (that optionally could output to the transcript)

Transcript can use stdout already. Not sure we want to shift the problem to the user again ... :-/ "Every time you provide an option, you expect users to make a decision." :-)

That's not what I meant.

Transcript is Smalltalk for stdout, I would say. Complicating it to act like a proper logger is IMHO ill-directed.

Neither C printf nor Java System.out.println[1] nor (i think) Racket display are thread safe.

Great examples. Let's not do what they do. We can easily provide a thread-safe Transcript, so why not do so?

...

And I think this is ok.

Transcript shouldn't break when used concurrently, but I don't think it should do something sophisticated…

It does break now.

Yeah i see.

...

Transcript is not a good logger. We shouldn't pretend it is.

Maybe GemStone's object log is a neat abstraction. Thread-safe, object based, with time stamps.

Perhaps because Transcript is not a logger. It's just a place you can write things to which will appear somewhere the user can see it. It can do that trhing right or wrong. I think we all agree it does not do it right now.

agreed.

But I'd rather see Transcript dumbed down, not clevered up. -t

Levente

...

Best regards -Tobias

...

Best, Marcel

...

Am 28.01.2021 10:32:27 schrieb Tobias Pape das.linux@gmx.de: Hi apart from the debuggers, why bother making Transcript thread-safe in the first place? The Transcript is a mere output stream for debugging/logging etc., our stdout or stderr. A separate kind of log (that optionally could output to the transcript) would be more apt IMHO. Thread safety could be addressed there without complicating Transcript more… my 2ct -tobias

...

On 28. Jan 2021, at 09:05, Marcel Taeumel wrote:

...

Hi Levente. Thanks für clarification. We are talking about more than one problem here. :-) I did just address the issue with errors/debuggers happening due to UI process disruptions. You are also concerned about the order of contents in the Transcript's contents. Both are very important and required for claiming that it would be thread-safe. (Which reminds me that I find it very strange that the UI components (e.g. PluggableTextMorph) do also check the Transcript's internal characterLimit. I would expect an append-only behavior when a Transcript window is open.) Anyway, your proposal about a safelyNextPut(All): looks good. :-) +1 Best,

Marcel

...

Am 27.01.2021 23:05:21 schrieb Levente Uzonyi : >> Hi Marcel, >> On Tue, 26 Jan 2021, Marcel Taeumel wrote: >> > Hi Jaromir, > please take a look at Collections-mt.923 (inbox). Maybe those changes would satisfy all your current needs in this regard. :-) > >> > @all: Would Transcript now be thread-safe? Did I miss something? >> Not at all. The problem is not #endEntry but all the other >> methods writing to the stream e.g.: #nextPutAll:. The real solution is to replace the stream-like API with one that provides >> an easy way to write messages in a thread-safe way. E.g.: >> Transcript showStream: [ :stream | stream nextPutAll: 'Hello'; space; nextPutAll: ' World!'; cr ] >> Which guarantees that 'Hello World!' appears on the Transcript without >> being mixed with other messages. >> or >> Transcript show: 'Hello {1}!{2}' format: { 'World'. String cr }. >> The implementations could be e.g.: >> showStream: aBlock >> | string | string := String streamContents: aBlock. self safelyNextPutAll: string >> show: aString format: arguments >> | string | string := aString format: arguments. self safelyNextPutAll: string >> The last challenge is to implement #safelyNextPutAll: which involves >> making Transcript not be a TranscriptStream. Transcript should encapsulate the stream and use a Mutex (not a global one >> because its pointless) to provide thread-safety while writing the >> characters on it or reading its contents. E.g.: >> safelyNextPutAll: aString >> mutex critical: [ stream nextPutAll: aString ] >> For backwards compatibility, the stream-API methods must be provided for a >> while but those methods should be thread-safe on their own. E.g.: >> nextPutAll: aString >> self safelyNextPutAll: aString >> nextPut: aCharacter >> self safelyNextPutAll: aCharacter asString >> >> Levente >> > >> > Best, > Marcel > > Am 25.01.2021 21:38:36 schrieb jaromir : > > Well, I tried deferring the whole Transcript endEntry machinery to the UI > doOneCycle (bypassing the changed: #appendEntryLater mechanism) for > #forceUpdate = false only ... and it seems to avoid the problem! > > TranscriptStream >> endEntry > > deferredEntry ifNil: [ false ]. "this is a new instance variable" > self semaphore critical:[ > self class forceUpdate > ifTrue: [self changed: #appendEntry; reset] > ifFalse: [deferredEntry := true]. > >> > > TranscriptStream >> flushDeferredEntry > "This is run every UI cycle in doOneCycleNowFor:" > > deferredEntry ifTrue: [ > self class forceUpdate: true. > self endEntry. > deferredEntry := false. > self class forceUpdate: false. > ] > > doOneCycleNowFor: aWorld > > "... the whole body remains unchanged except:" > > capturingGesture ifFalse: > [aWorld runStepMethods. > Transcript flushDeferredEntry. "this is printing for #forceUpdate = > false" > self displayWorldSafely: aWorld]. > >> > > For #forceUpdate = true the endEntry mechanism remains unchanged and failing > as before... > >> > >> > > -- > Sent from: http://forum.world.st/Squeak-Dev-f45488.html > >> > >> > >

Hi Christoph.

What is that dangerous thing that could happen during a suspension point in the VM? What am I missing here? :-)

It's only part of the problem. There could be a disconnect between the former "Project current uiProcess" comparison and how "addDeferredUIMessage" is processed later in the code. It would be better to no preempt this call as a whole.

Best, Marcel Am 28.01.2021 10:59:50 schrieb Thiede, Christoph christoph.thiede@student.hpi.uni-potsdam.de: Hi all,

this is an interesting discussion, two thoughts from my side:

1. What is the problem with "Processor activeProcess == Project current uiProcess" not being a thread-safe expression? I see that it could be possible at many points that the VM interrupts the current process and resumes another one first; but still, activeProcess will always hold the same Process instance when the same process is active, independently of any interruptions, won't it? What is that dangerous thing that could happen during a suspension point in the VM? What am I missing here? :-)

2. To Levente's proposal of making Transcript calls thread-safe:

safelyNextPutAll: aString             mutex critical: [                   stream nextPutAll: aString ]

So this would delay the execution of any process that uses the Transcript, wouldn't it? Hm ... I would rather have expected the Transcript calls to be deferred when the Transcript is locked ATM - *unless* #forceUpdate is enabled, maybe. Let's keep everyday Transcript clients blocking-free whenever possible. Or imagine several processes writing to the Transcript, but the transcript not being opened at this time. There should no need to block any of the calling processes?

Best, Christoph Von: Squeak-dev squeak-dev-bounces@lists.squeakfoundation.org im Auftrag von Taeumel, Marcel Gesendet: Donnerstag, 28. Januar 2021 10:41:03 An: squeak-dev Betreff: Re: [squeak-dev] Transcript error when forceUpdate: false (?)   Hi Tobias

A separate kind of log (that optionally could output to the transcript)

Transcript can use stdout already. Not sure we want to shift the problem to the user again ... :-/ "Every time you provide an option, you expect users to make a decision." :-)

Best, Marcel Am 28.01.2021 10:32:27 schrieb Tobias Pape das.linux@gmx.de: Hi

apart from the debuggers, why bother making Transcript thread-safe in the first place?

The Transcript is a mere output stream for debugging/logging etc., our stdout or stderr.

A separate kind of log (that optionally could output to the transcript) would be more apt IMHO. Thread safety could be addressed there without complicating Transcript more…

my 2ct -tobias

On 28. Jan 2021, at 09:05, Marcel Taeumel wrote:

Hi Levente.

Thanks für clarification. We are talking about more than one problem here. :-) I did just address the issue with errors/debuggers happening due to UI process disruptions. You are also concerned about the order of contents in the Transcript's contents. Both are very important and required for claiming that it would be thread-safe.

(Which reminds me that I find it very strange that the UI components (e.g. PluggableTextMorph) do also check the Transcript's internal characterLimit. I would expect an append-only behavior when a Transcript window is open.)

Anyway, your proposal about a safelyNextPut(All): looks good. :-) +1

Best, Marcel

...

Am 27.01.2021 23:05:21 schrieb Levente Uzonyi :

Hi Marcel,

On Tue, 26 Jan 2021, Marcel Taeumel wrote:

...

Hi Jaromir, please take a look at Collections-mt.923 (inbox). Maybe those changes would satisfy all your current needs in this regard. :-)

@all: Would Transcript now be thread-safe? Did I miss something?

Not at all. The problem is not #endEntry but all the other methods writing to the stream e.g.: #nextPutAll:. The real solution is to replace the stream-like API with one that provides an easy way to write messages in a thread-safe way. E.g.:

Transcript showStream: [ :stream | stream nextPutAll: 'Hello'; space; nextPutAll: ' World!'; cr ]

Which guarantees that 'Hello World!' appears on the Transcript without being mixed with other messages.

or

Transcript show: 'Hello {1}!{2}' format: { 'World'. String cr }.

The implementations could be e.g.:

showStream: aBlock

| string | string := String streamContents: aBlock. self safelyNextPutAll: string

show: aString format: arguments

| string | string := aString format: arguments. self safelyNextPutAll: string

The last challenge is to implement #safelyNextPutAll: which involves making Transcript not be a TranscriptStream. Transcript should encapsulate the stream and use a Mutex (not a global one because its pointless) to provide thread-safety while writing the characters on it or reading its contents. E.g.:

safelyNextPutAll: aString

mutex critical: [ stream nextPutAll: aString ]

For backwards compatibility, the stream-API methods must be provided for a while but those methods should be thread-safe on their own. E.g.:

nextPutAll: aString

self safelyNextPutAll: aString

nextPut: aCharacter

self safelyNextPutAll: aCharacter asString

Levente

...

Best, Marcel

Am 25.01.2021 21:38:36 schrieb jaromir :

Well, I tried deferring the whole Transcript endEntry machinery to the UI doOneCycle (bypassing the changed: #appendEntryLater mechanism) for #forceUpdate = false only ... and it seems to avoid the problem!

TranscriptStream >> endEntry

deferredEntry ifNil: [ false ]. "this is a new instance variable" self semaphore critical:[ self class forceUpdate ifTrue: [self changed: #appendEntry; reset] ifFalse: [deferredEntry := true].

TranscriptStream >> flushDeferredEntry "This is run every UI cycle in doOneCycleNowFor:"

deferredEntry ifTrue: [ self class forceUpdate: true. self endEntry. deferredEntry := false. self class forceUpdate: false. ]

doOneCycleNowFor: aWorld

"... the whole body remains unchanged except:"

capturingGesture ifFalse: [aWorld runStepMethods. Transcript flushDeferredEntry. "this is printing for #forceUpdate = false" self displayWorldSafely: aWorld].

For #forceUpdate = true the endEntry mechanism remains unchanged and failing as before...

-- Sent from: http://forum.world.st/Squeak-Dev-f45488.html

I still don't see the point. Are you saying that the current project or it's UI process could change during a VM suspension point?

If that code runs at 39 and the user switches projects at 40, yes, this can happen.

Then, what about something like "Project current evaluateUIMessage: [...]", which will unpreemptedly (using the new primitive solution) check the UI process and either evaluate the message directly or defer it? :-)

Even the implementation of #evaluateUIMessage: would need something like <noContextSwitch> :-) So, one could directly add this pragma to, for example, #endEntry, without adding such a new indirection. And at lower levels or more concise scopes, access should be controlled with a mutex.

I suppose that the "trick" with <noContextSwitch> only works in a green-threading model where exactly 1 thread (or Squeak process) is actually running at a time.

Best, Marcel

P.S.: Is every fallback code for failing primitives evaluated without preemption?  Am 28.01.2021 11:28:24 schrieb Thiede, Christoph christoph.thiede@student.hpi.uni-potsdam.de: I still don't see the point. Are you saying that the current project or it's UI process could change during a VM suspension point?

Then, what about something like "Project current evaluateUIMessage: [...]", which will unpreemptedly (using the new primitive solution) check the UI process and either evaluate the message directly or defer it? :-)

Best, Christoph [http://www.hpi.de/] Von: Squeak-dev squeak-dev-bounces@lists.squeakfoundation.org im Auftrag von Taeumel, Marcel Gesendet: Donnerstag, 28. Januar 2021 11:13:55 An: squeak-dev Betreff: Re: [squeak-dev] Transcript error when forceUpdate: false (?)   Hi Christoph.

What is that dangerous thing that could happen during a suspension point in the VM? What am I missing here? :-)

It's only part of the problem. There could be a disconnect between the former "Project current uiProcess" comparison and how "addDeferredUIMessage" is processed later in the code. It would be better to no preempt this call as a whole.

Best, Marcel Am 28.01.2021 10:59:50 schrieb Thiede, Christoph christoph.thiede@student.hpi.uni-potsdam.de: Hi all,

this is an interesting discussion, two thoughts from my side:

1. What is the problem with "Processor activeProcess == Project current uiProcess" not being a thread-safe expression? I see that it could be possible at many points that the VM interrupts the current process and resumes another one first; but still, activeProcess will always hold the same Process instance when the same process is active, independently of any interruptions, won't it? What is that dangerous thing that could happen during a suspension point in the VM? What am I missing here? :-)

2. To Levente's proposal of making Transcript calls thread-safe:

safelyNextPutAll: aString             mutex critical: [                   stream nextPutAll: aString ]

So this would delay the execution of any process that uses the Transcript, wouldn't it? Hm ... I would rather have expected the Transcript calls to be deferred when the Transcript is locked ATM - *unless* #forceUpdate is enabled, maybe. Let's keep everyday Transcript clients blocking-free whenever possible. Or imagine several processes writing to the Transcript, but the transcript not being opened at this time. There should no need to block any of the calling processes?

Best, Christoph Von: Squeak-dev squeak-dev-bounces@lists.squeakfoundation.org im Auftrag von Taeumel, Marcel Gesendet: Donnerstag, 28. Januar 2021 10:41:03 An: squeak-dev Betreff: Re: [squeak-dev] Transcript error when forceUpdate: false (?)   Hi Tobias

A separate kind of log (that optionally could output to the transcript)

Transcript can use stdout already. Not sure we want to shift the problem to the user again ... :-/ "Every time you provide an option, you expect users to make a decision." :-)

Best, Marcel Am 28.01.2021 10:32:27 schrieb Tobias Pape das.linux@gmx.de: Hi

apart from the debuggers, why bother making Transcript thread-safe in the first place?

The Transcript is a mere output stream for debugging/logging etc., our stdout or stderr.

A separate kind of log (that optionally could output to the transcript) would be more apt IMHO. Thread safety could be addressed there without complicating Transcript more…

my 2ct -tobias

On 28. Jan 2021, at 09:05, Marcel Taeumel wrote:

Hi Levente.

Thanks für clarification. We are talking about more than one problem here. :-) I did just address the issue with errors/debuggers happening due to UI process disruptions. You are also concerned about the order of contents in the Transcript's contents. Both are very important and required for claiming that it would be thread-safe.

(Which reminds me that I find it very strange that the UI components (e.g. PluggableTextMorph) do also check the Transcript's internal characterLimit. I would expect an append-only behavior when a Transcript window is open.)

Anyway, your proposal about a safelyNextPut(All): looks good. :-) +1

Best, Marcel

...

Am 27.01.2021 23:05:21 schrieb Levente Uzonyi :

Hi Marcel,

On Tue, 26 Jan 2021, Marcel Taeumel wrote:

...

Hi Jaromir, please take a look at Collections-mt.923 (inbox). Maybe those changes would satisfy all your current needs in this regard. :-)

@all: Would Transcript now be thread-safe? Did I miss something?

Not at all. The problem is not #endEntry but all the other methods writing to the stream e.g.: #nextPutAll:. The real solution is to replace the stream-like API with one that provides an easy way to write messages in a thread-safe way. E.g.:

Transcript showStream: [ :stream | stream nextPutAll: 'Hello'; space; nextPutAll: ' World!'; cr ]

Which guarantees that 'Hello World!' appears on the Transcript without being mixed with other messages.

or

Transcript show: 'Hello {1}!{2}' format: { 'World'. String cr }.

The implementations could be e.g.:

showStream: aBlock

| string | string := String streamContents: aBlock. self safelyNextPutAll: string

show: aString format: arguments

| string | string := aString format: arguments. self safelyNextPutAll: string

The last challenge is to implement #safelyNextPutAll: which involves making Transcript not be a TranscriptStream. Transcript should encapsulate the stream and use a Mutex (not a global one because its pointless) to provide thread-safety while writing the characters on it or reading its contents. E.g.:

safelyNextPutAll: aString

mutex critical: [ stream nextPutAll: aString ]

For backwards compatibility, the stream-API methods must be provided for a while but those methods should be thread-safe on their own. E.g.:

nextPutAll: aString

self safelyNextPutAll: aString

nextPut: aCharacter

self safelyNextPutAll: aCharacter asString

Levente

...

Best, Marcel

Am 25.01.2021 21:38:36 schrieb jaromir :

Well, I tried deferring the whole Transcript endEntry machinery to the UI doOneCycle (bypassing the changed: #appendEntryLater mechanism) for #forceUpdate = false only ... and it seems to avoid the problem!

TranscriptStream >> endEntry

deferredEntry ifNil: [ false ]. "this is a new instance variable" self semaphore critical:[ self class forceUpdate ifTrue: [self changed: #appendEntry; reset] ifFalse: [deferredEntry := true].

TranscriptStream >> flushDeferredEntry "This is run every UI cycle in doOneCycleNowFor:"

deferredEntry ifTrue: [ self class forceUpdate: true. self endEntry. deferredEntry := false. self class forceUpdate: false. ]

doOneCycleNowFor: aWorld

"... the whole body remains unchanged except:"

capturingGesture ifFalse: [aWorld runStepMethods. Transcript flushDeferredEntry. "this is printing for #forceUpdate = false" self displayWorldSafely: aWorld].

For #forceUpdate = true the endEntry mechanism remains unchanged and failing as before...

-- Sent from: http://forum.world.st/Squeak-Dev-f45488.html

Hi Christoph,

This sounds a very bit speculative to me ... Is it really worth the raised complexity? :-)

We are talking about whether <noContextSwitch> could help to make code thread-safe where a mutex might be not sufficient or possible.

Whether or not this is useful to the average Smalltalk programmer using Transcript is pure speculation. The underlying issue is, however, real. :-) Now, it depends, how critical the systems are that you are developing. So, for people that want to process critical data, Transcript must work ... or an alternative logging mechanism has to be created for that purpose. This leads to the discussion about the features the underlying framework/library should provide ... ;-)

Best, Marcel Am 01.02.2021 15:29:14 schrieb Thiede, Christoph christoph.thiede@student.hpi.uni-potsdam.de: Hi Marcel,

thanks for the explanation. Just to make sure I understand correctly: We are talking about maybe changing the transcript interface and/or even introducing a new pragma selector solely for the purpose of smoothly handling this super-edge case of the active project being switched just while an always-running background thread is logging something to the Transcript? This sounds a very bit speculative to me ... Is it really worth the raised complexity? :-)

Best, Christoph [http://www.hpi.de/] Von: Squeak-dev squeak-dev-bounces@lists.squeakfoundation.org im Auftrag von Taeumel, Marcel Gesendet: Donnerstag, 28. Januar 2021 12:14:11 An: squeak-dev Betreff: Re: [squeak-dev] Transcript error when forceUpdate: false (?)

I still don't see the point. Are you saying that the current project or it's UI process could change during a VM suspension point?

If that code runs at 39 and the user switches projects at 40, yes, this can happen.

Then, what about something like "Project current evaluateUIMessage: [...]", which will unpreemptedly (using the new primitive solution) check the UI process and either evaluate the message directly or defer it? :-)

Even the implementation of #evaluateUIMessage: would need something like <noContextSwitch> :-) So, one could directly add this pragma to, for example, #endEntry, without adding such a new indirection. And at lower levels or more concise scopes, access should be controlled with a mutex.

I suppose that the "trick" with <noContextSwitch> only works in a green-threading model where exactly 1 thread (or Squeak process) is actually running at a time.

Best, Marcel

P.S.: Is every fallback code for failing primitives evaluated without preemption?  Am 28.01.2021 11:28:24 schrieb Thiede, Christoph christoph.thiede@student.hpi.uni-potsdam.de: I still don't see the point. Are you saying that the current project or it's UI process could change during a VM suspension point?

Then, what about something like "Project current evaluateUIMessage: [...]", which will unpreemptedly (using the new primitive solution) check the UI process and either evaluate the message directly or defer it? :-)

Best, Christoph [http://www.hpi.de/] Von: Squeak-dev squeak-dev-bounces@lists.squeakfoundation.org im Auftrag von Taeumel, Marcel Gesendet: Donnerstag, 28. Januar 2021 11:13:55 An: squeak-dev Betreff: Re: [squeak-dev] Transcript error when forceUpdate: false (?)   Hi Christoph.

What is that dangerous thing that could happen during a suspension point in the VM? What am I missing here? :-)

It's only part of the problem. There could be a disconnect between the former "Project current uiProcess" comparison and how "addDeferredUIMessage" is processed later in the code. It would be better to no preempt this call as a whole.

Best, Marcel Am 28.01.2021 10:59:50 schrieb Thiede, Christoph christoph.thiede@student.hpi.uni-potsdam.de: Hi all,

this is an interesting discussion, two thoughts from my side:

1. What is the problem with "Processor activeProcess == Project current uiProcess" not being a thread-safe expression? I see that it could be possible at many points that the VM interrupts the current process and resumes another one first; but still, activeProcess will always hold the same Process instance when the same process is active, independently of any interruptions, won't it? What is that dangerous thing that could happen during a suspension point in the VM? What am I missing here? :-)

2. To Levente's proposal of making Transcript calls thread-safe:

safelyNextPutAll: aString             mutex critical: [                   stream nextPutAll: aString ]

So this would delay the execution of any process that uses the Transcript, wouldn't it? Hm ... I would rather have expected the Transcript calls to be deferred when the Transcript is locked ATM - *unless* #forceUpdate is enabled, maybe. Let's keep everyday Transcript clients blocking-free whenever possible. Or imagine several processes writing to the Transcript, but the transcript not being opened at this time. There should no need to block any of the calling processes?

Best, Christoph Von: Squeak-dev squeak-dev-bounces@lists.squeakfoundation.org im Auftrag von Taeumel, Marcel Gesendet: Donnerstag, 28. Januar 2021 10:41:03 An: squeak-dev Betreff: Re: [squeak-dev] Transcript error when forceUpdate: false (?)   Hi Tobias

A separate kind of log (that optionally could output to the transcript)

Transcript can use stdout already. Not sure we want to shift the problem to the user again ... :-/ "Every time you provide an option, you expect users to make a decision." :-)

Best, Marcel Am 28.01.2021 10:32:27 schrieb Tobias Pape das.linux@gmx.de: Hi

apart from the debuggers, why bother making Transcript thread-safe in the first place?

The Transcript is a mere output stream for debugging/logging etc., our stdout or stderr.

A separate kind of log (that optionally could output to the transcript) would be more apt IMHO. Thread safety could be addressed there without complicating Transcript more…

my 2ct -tobias

On 28. Jan 2021, at 09:05, Marcel Taeumel wrote:

Hi Levente.

Thanks für clarification. We are talking about more than one problem here. :-) I did just address the issue with errors/debuggers happening due to UI process disruptions. You are also concerned about the order of contents in the Transcript's contents. Both are very important and required for claiming that it would be thread-safe.

(Which reminds me that I find it very strange that the UI components (e.g. PluggableTextMorph) do also check the Transcript's internal characterLimit. I would expect an append-only behavior when a Transcript window is open.)

Anyway, your proposal about a safelyNextPut(All): looks good. :-) +1

Best, Marcel

...

Am 27.01.2021 23:05:21 schrieb Levente Uzonyi :

Hi Marcel,

On Tue, 26 Jan 2021, Marcel Taeumel wrote:

...

Hi Jaromir, please take a look at Collections-mt.923 (inbox). Maybe those changes would satisfy all your current needs in this regard. :-)

@all: Would Transcript now be thread-safe? Did I miss something?

Not at all. The problem is not #endEntry but all the other methods writing to the stream e.g.: #nextPutAll:. The real solution is to replace the stream-like API with one that provides an easy way to write messages in a thread-safe way. E.g.:

Transcript showStream: [ :stream | stream nextPutAll: 'Hello'; space; nextPutAll: ' World!'; cr ]

Which guarantees that 'Hello World!' appears on the Transcript without being mixed with other messages.

or

Transcript show: 'Hello {1}!{2}' format: { 'World'. String cr }.

The implementations could be e.g.:

showStream: aBlock

| string | string := String streamContents: aBlock. self safelyNextPutAll: string

show: aString format: arguments

| string | string := aString format: arguments. self safelyNextPutAll: string

The last challenge is to implement #safelyNextPutAll: which involves making Transcript not be a TranscriptStream. Transcript should encapsulate the stream and use a Mutex (not a global one because its pointless) to provide thread-safety while writing the characters on it or reading its contents. E.g.:

safelyNextPutAll: aString

mutex critical: [ stream nextPutAll: aString ]

For backwards compatibility, the stream-API methods must be provided for a while but those methods should be thread-safe on their own. E.g.:

nextPutAll: aString

self safelyNextPutAll: aString

nextPut: aCharacter

self safelyNextPutAll: aCharacter asString

Levente

...

Best, Marcel

Am 25.01.2021 21:38:36 schrieb jaromir :

Well, I tried deferring the whole Transcript endEntry machinery to the UI doOneCycle (bypassing the changed: #appendEntryLater mechanism) for #forceUpdate = false only ... and it seems to avoid the problem!

TranscriptStream >> endEntry

deferredEntry ifNil: [ false ]. "this is a new instance variable" self semaphore critical:[ self class forceUpdate ifTrue: [self changed: #appendEntry; reset] ifFalse: [deferredEntry := true].

TranscriptStream >> flushDeferredEntry "This is run every UI cycle in doOneCycleNowFor:"

deferredEntry ifTrue: [ self class forceUpdate: true. self endEntry. deferredEntry := false. self class forceUpdate: false. ]

doOneCycleNowFor: aWorld

"... the whole body remains unchanged except:"

capturingGesture ifFalse: [aWorld runStepMethods. Transcript flushDeferredEntry. "this is printing for #forceUpdate = false" self displayWorldSafely: aWorld].

For #forceUpdate = true the endEntry mechanism remains unchanged and failing as before...

-- Sent from: http://forum.world.st/Squeak-Dev-f45488.html

Hi Marcel, Hi All,

On Mon, Jan 25, 2021 at 3:26 AM Marcel Taeumel marcel.taeumel@hpi.de wrote:

...

It's simply not thread-safe.

Hmm... in particular, there is a side effect (i.e. layout update) in the window's pluggable text morph. Even if #forceUpdate is false.

In #endEntry, we could check for "Processor activeProcess == Project current uiProcess" to then use "Project current addDeferredUIMessage..." automatically.

That gave me a scare. At first I thought that the effectiveProcess changes to ProcessorScheduler>>activeProcess could have made that expression no longer thread safe. But it isn't anyway, and the above is very bad style. Let me explain.

The suspension points in the VM are - the first bytecode of any non-quick method (following a primitive if it has one) - the first bytecode of any block - a backward branch

In the JIT there are fewer suspension points: - the first bytecode of any method containing a send other than #== and/or #class (and given that ifNil:ifNotNil: is inlined into a send of #== and jumps, ifNil: sends don't count) - the first bytecode of any block containing a send other than #== and/or #class - a backward branch

So given

ProcessorScheduler>>#activeProcess "suspension point here on an interpreter VM, and the first time in the JIT VM" ^activeProcess effectiveProcess

Process>>#effectiveProcess "suspension point here on an interpreter VM, and the first time in the JIT VM" ^effectiveProcess ifNil: [self]

Project class>>#current "suspension point here on an interpreter VM, and the first time in the JIT VM" ^CurrentProject

MVCProject>>#uiProcess "suspension point here on an interpreter VM, and the first time in the JIT VM" ^ world activeControllerProcess

MorphicProject>>#uiProcess "no suspension point here on any VM obeying the blue-book spec" ^uiProcess

we can see that "Processor activeProcess == Project current uiProcess" is not thread-safe in the interpreter; there is a suspension point in Project current before CurrentProject is returned (but not in the JIT); there is a suspension point in MVC in uiProcess. There is a suspension point in ProcessorScheduler>>#activeProcess before the send of effectiveProcess to activeProcess, but since no data is fetched until the process resumes this is harmless. However, were the expression written as "Project current uiProcess == Processor activeProcess" then this would no longer be harmless.

So on all VMs this is not reliably thread-safe. Even on the JIT VM the first use of any method (except a doit) is interpreted. Once the JIT VM, once it has started up and the methods have been jitted, "Processor activeProcess == Project current uiProcess" is thread-safe on Morphic, but not on MVC. So we really shouldn't be using things like this. They'll appear to work and in very rare circumstances they won't. So let's please use the proper construct, Mutex.

But that makes me realize that the effectiveProcess changes are unsafe on an interpreter VM (and hence we're on;y getting away with it on the JIT VM). Se really need some way of saying that ProcessorScheduler>>#activeProcess is atomic, e.g.

Process>>#effectiveProcess <noContextSwitch> ^effectiveProcess ifNil: [self]

But I don't know how to implement this right now. Note that we do have

BlockClosure>>#valueNoContextSwitch "An exact copy of BlockClosure>>value except that this version will not preempt the current process on block activation if a higher-priority process is runnable. Primitive. Essential." <primitive: 221> numArgs ~= 0 ifTrue: [self numArgsError: 0]. self primitiveFailed

FullBlockClosure>>#valueNoContextSwitch "An exact copy of BlockClosure>>value except that this version will not preempt the current process on block activation if a higher-priority process is runnable. Primitive. Essential." <primitive: 209> numArgs ~= 0 ifTrue: [self numArgsError: 0]. self primitiveFailed

but the VM uses the primitive numbers to control context switching. We don't have a mechanism the VM can use to label a normal method such as Process>>#effectiveProcess as not to allow an interrupt.

Best, Marcel

Am 23.01.2021 21:05:05 schrieb Levente Uzonyi leves@caesar.elte.hu: On Fri, 22 Jan 2021, tim Rowledge wrote:

...

...

On 2021-01-22, at 5:26 PM, David T. Lewis wrote:

On Fri, Jan 22, 2021 at 02:03:10PM -0600, jaromir wrote:

...

I ran the following in Squeak 5.3:

TranscriptStream forceUpdate: false. [ 10000 timesRepeat: [ Transcript show: 'x' ] ] forkAt: 39

When the Transcript window fills up, 'Message not understood' and

'Assertion

...

...

...

failure' appear. I'm wondering is this a bug? Thanks. Jaromir

It certainly does appear to be a bug.

We had some discussion about it not so long ago; you can't use the

Transcript to log bugs from Seaside very well, for example. Background processes, mutexs, that sort of thing. IIRC Levente offered an explanation?

It's simply not thread-safe. If you want it to be thread-safe, you can pass your message to the UI process to show it. Make sure your message is computed in its own process and is passed as a precomputed string to avoid other kinds of race conditions. E.g.:

| theMessage | theMessage := 'Something very {1}.' format: { 'important' }. Project current addDeferredUIMessage: [ Transcript show: theMessage; cr ].

Levente

...

tim

tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim The hardness of the butter is proportional to the softness of the bread.

Hi Eliot.

Thanks for the clarification of VM suspension points.

So let's please use the proper construct, Mutex.

Hmmm. I suppose it's not that easy. If #endEntry is running inside the ui process, we must not wait to not block the ui process (given that we want to implement #forceUpdate) -- but instead complete all deferred messages or just trigger a drawing cycle. A simple mutex wouldn't do because we want to avoid context switching altogether while executing methods that use such process comparision.

Thread-safety might be an issue in several places, which kind of use "Processor activeProcess == Project current uiProcess":

MorphicProject >> #spawnNewProcessIfThisIsUI: (Morphic|MVC)Debugger class >> #openOn:...

Even there, a mutex would not do because we want to avoid comparing the wrong objects, not to avoid reentering that section. The entire expression "Processor activeProcess == Project current uiProcess" should be atomic.

Any unused numeric primitive could be used as a marker to avoid context switches.  I propose that we use it, hiding it behind a suitable pragma for syntactic sugar (i.e. <noContextSwitch>).  123 used to be primitiveValueUninteruptibly, which we no longer use.  I propose we use this.

+1 :-)

Best, Marcel Am 27.01.2021 01:35:09 schrieb Eliot Miranda eliot.miranda@gmail.com:

On Tue, Jan 26, 2021 at 2:45 PM Eliot Miranda <eliot.miranda@gmail.com [mailto:eliot.miranda@gmail.com]> wrote:

Hi Marcel, Hi All,

On Mon, Jan 25, 2021 at 3:26 AM Marcel Taeumel <marcel.taeumel@hpi.de [mailto:marcel.taeumel@hpi.de]> wrote:

It's simply not thread-safe.

Hmm... in particular, there is a side effect (i.e. layout update) in the window's pluggable text morph. Even if #forceUpdate is false.

In #endEntry, we could check for "Processor activeProcess == Project current uiProcess" to then use "Project current addDeferredUIMessage..." automatically.

That gave me a scare.  At first I thought that the effectiveProcess changes to ProcessorScheduler>>activeProcess could have made that expression no longer thread safe.  But it isn't anyway, and the above is very bad style.  Let me explain.

The suspension points in the VM are  - the first bytecode of any non-quick method (following a primitive if it has one) - the first bytecode of any block - a backward branch

In the JIT there are fewer suspension points: - the first bytecode of any method containing a send other than #== and/or #class (and given that ifNil:ifNotNil: is inlined into a send of #== and jumps, ifNil: sends don't count) - the first bytecode of any block containing a send other than #== and/or #class - a backward branch

So given

ProcessorScheduler>>#activeProcess         "suspension point here on an interpreter VM, and the first time in the JIT VM" ^activeProcess effectiveProcess

Process>>#effectiveProcess         "suspension point here on an interpreter VM, and the first time in the JIT VM" ^effectiveProcess ifNil: [self]

Project class>>#current         "suspension point here on an interpreter VM, and the first time in the JIT VM" ^CurrentProject

MVCProject>>#uiProcess

"suspension point here on an interpreter VM, and the first time in the JIT VM" ^ world activeControllerProcess

MorphicProject>>#uiProcess         "no suspension point here on any VM obeying the blue-book spec" ^uiProcess

we can see that "Processor activeProcess == Project current uiProcess" is not thread-safe in the interpreter; there is a suspension point in Project current before CurrentProject is returned (but not in the JIT); there is a suspension point in MVC in uiProcess.  There is a suspension point in ProcessorScheduler>>#activeProcess before the send of effectiveProcess to activeProcess, but since no data is fetched until the process resumes this is harmless.  However, were the expression written as "Project current uiProcess == Processor activeProcess" then this would no longer be harmless.

So on all VMs this is not reliably thread-safe.  Even on the JIT VM the first use of any method (except a doit) is interpreted.  Once the JIT VM, once it has started up and the methods have been jitted, "Processor activeProcess == Project current uiProcess" is thread-safe on Morphic, but not on MVC.  So we really shouldn't be using things like this.  They'll appear to work and in very rare circumstances they won't.  So let's please use the proper construct, Mutex.

But that makes me realize that the effectiveProcess changes are unsafe on an interpreter VM (and hence we're on;y getting away with it on the JIT VM).  Se really need some way of saying that ProcessorScheduler>>#activeProcess is atomic, e.g.

Process>>#effectiveProcess         <noContextSwitch> ^effectiveProcess ifNil: [self]

But I don't know how to implement this right now.  Note that we do have

BlockClosure>>#valueNoContextSwitch "An exact copy of BlockClosure>>value except that this version will not preempt the current process on block activation if a higher-priority process is runnable. Primitive. Essential." <primitive: 221> numArgs ~= 0 ifTrue: [self numArgsError: 0]. self primitiveFailed

FullBlockClosure>>#valueNoContextSwitch "An exact copy of BlockClosure>>value except that this version will not preempt the current process on block activation if a higher-priority process is runnable. Primitive. Essential." <primitive: 209> numArgs ~= 0 ifTrue: [self numArgsError: 0]. self primitiveFailed

but the VM uses the primitive numbers to control context switching.  We don't have a mechanism the VM can use to label a normal method such as Process>>#effectiveProcess as not to allow an interrupt.

Doh!  We have primitive numbers.  We already use primitive 19 as a "simulation guard" in the debugger machinery as a "simulation guard" to avoid I'm not sure what.  Any unused numeric primitive could be used as a marker to avoid context switches.  I propose that we use it, hiding it behind a suitable pragma for syntactic sugar (i.e. <noContextSwitch>).  123 used to be primitiveValueUninteruptibly, which we no longer use.  I propose we use this.

Best,

Marcel Am 23.01.2021 21:05:05 schrieb Levente Uzonyi <leves@caesar.elte.hu [mailto:leves@caesar.elte.hu]>: On Fri, 22 Jan 2021, tim Rowledge wrote:

...

On 2021-01-22, at 5:26 PM, David T. Lewis wrote:

On Fri, Jan 22, 2021 at 02:03:10PM -0600, jaromir wrote:

...

I ran the following in Squeak 5.3:

TranscriptStream forceUpdate: false. [ 10000 timesRepeat: [ Transcript show: 'x' ] ] forkAt: 39

When the Transcript window fills up, 'Message not understood' and 'Assertion failure' appear. I'm wondering is this a bug? Thanks. Jaromir

It certainly does appear to be a bug.

We had some discussion about it not so long ago; you can't use the Transcript to log bugs from Seaside very well, for example. Background processes, mutexs, that sort of thing. IIRC Levente offered an explanation?

It's simply not thread-safe. If you want it to be thread-safe, you can pass your message to the UI process to show it. Make sure your message is computed in its own process and is passed as a precomputed string to avoid other kinds of race conditions. E.g.:

| theMessage | theMessage := 'Something very {1}.' format: { 'important' }. Project current addDeferredUIMessage: [ Transcript show: theMessage; cr ].

Levente

tim

tim Rowledge; tim@rowledge.org [mailto:tim@rowledge.org]; http://www.rowledge.org/tim [http://www.rowledge.org/tim] The hardness of the butter is proportional to the softness of the bread.

--

_,,,^..^,,,_

best, Eliot

--

_,,,^..^,,,_

best, Eliot