Andreas, I said:
-----Original Message----- From: Withers, Robert [mailto:rwithers@quallaby.com]
I am curious about your use the ivar #myQueue in the ScriptProcess. I really like that your ScriptProcess will compute a value, and that that value is the 'result' of the process. That's a very standard view of computations. Isn't it a functional view? A better approach for supporting eventual sending may be to have a subclass of ScriptProcess which holds the continuation to the invocation site.
Actually, I am now thinking that a subclass may not be needed, since ScriptProcess already holds onto the invoking message (AsyncScriptMessageSend to be replaced by EventualScriptMessageSend for eventual sending) and the owner, which is the same object. There are also the ivars #myHandler and #myEventMap.
In the following method, called when the Process returns from the last stack frame, there are three different mechanisms that we could hook into to activate the resolver.
ScriptProcess>>#stopScriptProcess: result ... owner ifNotNil:[owner scriptStopped: self]. myHandler ifNotNil:[myHandler releaseTo: nil]. myEventMap ifNotNil:[self signal: #done with: resultObj].
- owner is the ScriptMessageSend, so an implementation of #scriptStopped: in the EventualScriptMessageSend, could activate the resolver.
- myHandler is set when sending #observe: and it is an EventObserver. Currently this only looks to be used by the #waitUntil protocols and it blocks on a semaphore. I suppose that a special observer could be created that holds the resolver.
- myEventMap holds event registrations. These can be set at a low-level with #on:notify: or at a higher level by startScript:when:. So after the EventualScriptMessageSend is converted to a ScriptProcess, a registration of the following could wake up the resolver proc on: #done notify: (#resolve: asAsyncScriptMessageIn: self resolver arguments: #())
What do you think would be the best way to activate the resolver?
cheers, rob
Rob,
What do you think would be the best way to activate the resolver?
Remember the resolver in the process' properties (cheap way to experiment with it - later we may want this to be an iVar) and then use #stopScriptProcess: to notify the resolver (if there is one). Seems like the simplest solution to me. One more thing to keep in mind though: #stopScriptProcess: is also invoked from #terminate and you'll probably need something else to notify the resolver that this promise was broken.
Cheers, - Andreas
-----Original Message----- From: squeak-e-bounces@lists.squeakfoundation.org [mailto:squeak-e-bounces@lists.squeakfoundation.org] On Behalf Of Withers, Robert Sent: Friday, February 14, 2003 6:30 PM To: 'Squeak-E - a capability-secure Squeak' Subject: RE: [Squeak-e] adding eventual sending to Croquet's ScriptProcess and ScriptScheduler
Andreas, I said:
-----Original Message----- From: Withers, Robert [mailto:rwithers@quallaby.com]
I am curious about your use the ivar #myQueue in the
ScriptProcess. I
really like that your ScriptProcess will compute a value, and that that value is the 'result' of the process. That's a very
standard view of
computations. Isn't it a functional view? A better approach for supporting eventual sending may be to have a subclass of ScriptProcess which holds the continuation to the invocation site.
Actually, I am now thinking that a subclass may not be needed, since ScriptProcess already holds onto the invoking message (AsyncScriptMessageSend to be replaced by EventualScriptMessageSend for eventual sending) and the owner, which is the same object. There are also the ivars #myHandler and #myEventMap.
In the following method, called when the Process returns from the last stack frame, there are three different mechanisms that we could hook into to activate the resolver.
ScriptProcess>>#stopScriptProcess: result ... owner ifNotNil:[owner scriptStopped: self]. myHandler ifNotNil:[myHandler releaseTo: nil]. myEventMap ifNotNil:[self signal: #done with: resultObj].
- owner is the ScriptMessageSend, so an implementation of
#scriptStopped: in the EventualScriptMessageSend, could activate the resolver.
- myHandler is set when sending #observe: and it is an EventObserver.
Currently this only looks to be used by the #waitUntil protocols and it blocks on a semaphore. I suppose that a special observer could be created that holds the resolver.
- myEventMap holds event registrations. These can be set at
a low-level with #on:notify: or at a higher level by startScript:when:. So after the EventualScriptMessageSend is converted to a ScriptProcess, a registration of the following could wake up the resolver proc on: #done notify: (#resolve: asAsyncScriptMessageIn: self resolver arguments: #())
What do you think would be the best way to activate the resolver?
cheers, rob _______________________________________________ Squeak-e mailing list Squeak-e@lists.squeakfoundation.org http://lists.squeakfoundation.org/listinfo/squeak-e
On Saturday, February 15, 2003, at 10:41 AM, Andreas Raab wrote:
Rob,
What do you think would be the best way to activate the resolver?
Remember the resolver in the process' properties (cheap way to experiment with it - later we may want this to be an iVar) and then use #stopScriptProcess: to notify the resolver (if there is one). Seems like the simplest solution to me. One more thing to keep in mind though: #stopScriptProcess: is also invoked from #terminate and you'll probably need something else to notify the resolver that this promise was broken.
Andreas, I have integrated eventual sending into the Scripting framework. It looks like there may be a reference issue with #waitUntil:, since I have to inspect the scriptProcess or the results are intermittent.
The easy way was an EventualScriptMessageSend, which holds the resolver. The #valueWithArguments:event: method computes the result and resolves the promise ( or smashes it).
If you load SqueakElib ( http://swiki.squeakfoundation.org/squeak-e/3 ) into a *throw away* Croquet image (I modify the Interpreter), then load this changeset: ( http://swiki.squeakfoundation.org/squeak-e/uploads/3/croquet-squeake- bridge.cs ).
You can look at the comment of CroquetContext for examples.
But...there is an issue:
The problem i am having is that I can't seem to get #waitUntil: to work. I need that so that a send of #immediateRef, to a promise, will result in that script blocking, until the underlying computations have completed and resolved the promise. This computes intermittent results. I believe that the ScriptProcess is going away. So, to test I tried building a test out of this:
[Transcript cr; show: 'platformName: '; show: ((Smalltalk startScript: #platformName) waitUntil: #done; result) ] newScript resume.
Otherwise it is working well, and I now have a "working" (minus the issue above) CroquetPromiseContext>>#immediateRef. :-))))
I am really impressed with the work you have here, Andreas. It's really powerful.
cheers! rob
Here is the performance dump of a regular smalltalk expression and an eventual smalltalk expression that is fairly intensive, but it only evaluates 4 eventual message sends. The cost of an eventual send is between 20 and 80 ms. I need to figure out a way to make these measurements more accurate.
The performance ratio is: 1.032
Output from straight smalltalk
Morphs with more than 50 instances Thumbnail => 118 PolygonMorph => 139 BorderedStringMorph => 62 AlignmentMorph => 253 ThreePhaseButtonMorph => 61 SameGameTile => 201 Morph => 267 TransformMorph => 58 StringMorph => 1379 TileMorph => 58 ImageMorph => 193 ScrollBar => 58 SketchMorph => 157 RectangleMorph => 611 IconicButton => 143 SimpleButtonMorph => 101 WordGameLetterMorph => 320 start time: 23985907 stop time: 23996228 running time: 10321 ms
Output from the eventualCroquetRef
Morphs with more than 50 instances Thumbnail => 118 PolygonMorph => 139 BorderedStringMorph => 62 AlignmentMorph => 252 ThreePhaseButtonMorph => 61 SameGameTile => 201 Morph => 267 TransformMorph => 55 StringMorph => 1361 TileMorph => 58 ImageMorph => 187 ScrollBar => 55 SketchMorph => 153 RectangleMorph => 588 IconicButton => 139 SimpleButtonMorph => 101 WordGameLetterMorph => 320 start time: 24012995 stop time: 24022998 running time: 10003 ms
On Saturday, February 15, 2003, at 09:41 PM, Robert Withers wrote:
You can look at the comment of CroquetContext for examples.
Oh, below is the code I ran for the performance measurements with the eventual sending. The eventual ref is created by sending #eventualCroquetRef. The eventual sends are: #withAllSubclasses, #select:, #inject:into:, #contents. #immediateRef sends #waitUntil:.
| start stop | start := Time millisecondClockValue. [Transcript cr; show: 'Morphs with more than 50 instances '; show: ((Morph eventualCroquetRef withAllSubclasses select: [:e | e allInstances size > 50]) inject: (WriteStream on: '') into: [:stream :class | stream cr; tab; tab; nextPutAll: class asString; nextPutAll: ' => '; nextPutAll: class allInstances size asString. stream]) contents immediateRef. stop := Time millisecondClockValue. Transcript cr; tab; show: 'start time: ', start asString; cr; tab; show: 'stop time: ', stop asString; cr; tab; show: 'running time: ', (stop - start) asString, ' ms'] newScript resume.
rob
*sigh*
I switched the results here. The corrected times are now listed, below. But anyway, it's really a very, stupid test of performance. More importantly, it shows that the code can look and feel superfluous when eventual sending. This was a very important goal of mine, if not the top goal, when I started to integrate eventual sending and squeak.
Now to we need to figure out how to do eventual exception handling. :)
cheers, rob
On Saturday, February 15, 2003, at 09:41 PM, Robert Withers wrote:
Output from straight smalltalk
running time: 10003 ms
Output from the eventualCroquetRef
running time: 10321 ms
Squeak-e mailing list Squeak-e@lists.squeakfoundation.org http://lists.squeakfoundation.org/listinfo/squeak-e
But...there is an issue:
Duh! Obsolete code! Sorry 'bout that - #newScript in both context and symbol is entirely obsolete and no longer used. You need to use #startScript: for everything, e.g.,
self startScript:
[Transcript cr; show: 'platformName: '; show: ((Smalltalk startScript: #platformName) waitUntil: #done; result) ]
will work.
Cheers, - Andreas
-----Original Message----- From: squeak-e-bounces@lists.squeakfoundation.org [mailto:squeak-e-bounces@lists.squeakfoundation.org] On Behalf Of Robert Withers Sent: Sunday, February 16, 2003 3:41 AM To: Squeak-E - a capability-secure Squeak Subject: Re: [Squeak-e] adding eventual sending to Croquet's ScriptProcess andScriptScheduler
On Saturday, February 15, 2003, at 10:41 AM, Andreas Raab wrote:
Rob,
What do you think would be the best way to activate the resolver?
Remember the resolver in the process' properties (cheap way to experiment with it - later we may want this to be an iVar) and then use #stopScriptProcess: to notify the resolver (if there is
one). Seems
like the simplest solution to me. One more thing to keep in mind though: #stopScriptProcess: is also invoked from #terminate and you'll probably need something else to notify the resolver that this promise was broken.
Andreas, I have integrated eventual sending into the Scripting framework. It looks like there may be a reference issue with #waitUntil:, since I have to inspect the scriptProcess or the results are intermittent.
The easy way was an EventualScriptMessageSend, which holds the resolver. The #valueWithArguments:event: method computes the result and resolves the promise ( or smashes it).
If you load SqueakElib ( http://swiki.squeakfoundation.org/squeak-e/3 ) into a *throw away* Croquet image (I modify the Interpreter), then load this changeset: ( http://swiki.squeakfoundation.org/squeak-e/uploads/3/croquet-squeake- bridge.cs ).
You can look at the comment of CroquetContext for examples.
But...there is an issue:
The problem i am having is that I can't seem to get #waitUntil: to work. I need that so that a send of #immediateRef, to a promise, will result in that script blocking, until the underlying computations have completed and resolved the promise. This computes intermittent results. I believe that the ScriptProcess is going away. So, to test I tried building a test out of this:
[Transcript cr; show: 'platformName: '; show: ((Smalltalk startScript: #platformName) waitUntil: #done; result) ] newScript resume.
Otherwise it is working well, and I now have a "working" (minus the issue above) CroquetPromiseContext>>#immediateRef. :-))))
I am really impressed with the work you have here, Andreas. It's really powerful.
cheers! rob
Here is the performance dump of a regular smalltalk expression and an eventual smalltalk expression that is fairly intensive, but it only evaluates 4 eventual message sends. The cost of an eventual send is between 20 and 80 ms. I need to figure out a way to make these measurements more accurate.
The performance ratio is: 1.032
Output from straight smalltalk
Morphs with more than 50 instances Thumbnail => 118 PolygonMorph => 139 BorderedStringMorph => 62 AlignmentMorph => 253 ThreePhaseButtonMorph => 61 SameGameTile => 201 Morph => 267 TransformMorph => 58 StringMorph => 1379 TileMorph => 58 ImageMorph => 193 ScrollBar => 58 SketchMorph => 157 RectangleMorph => 611 IconicButton => 143 SimpleButtonMorph => 101 WordGameLetterMorph => 320 start time: 23985907 stop time: 23996228 running time: 10321 ms
Output from the eventualCroquetRef
Morphs with more than 50 instances Thumbnail => 118 PolygonMorph => 139 BorderedStringMorph => 62 AlignmentMorph => 252 ThreePhaseButtonMorph => 61 SameGameTile => 201 Morph => 267 TransformMorph => 55 StringMorph => 1361 TileMorph => 58 ImageMorph => 187 ScrollBar => 55 SketchMorph => 153 RectangleMorph => 588 IconicButton => 139 SimpleButtonMorph => 101 WordGameLetterMorph => 320 start time: 24012995 stop time: 24022998 running time: 10003 ms
Squeak-e mailing list Squeak-e@lists.squeakfoundation.org http://lists.squeakfoundation.org/listinfo/squeak-e
On Sunday, February 16, 2003, at 08:09 AM, Andreas Raab wrote:
But...there is an issue:
Duh! Obsolete code! Sorry 'bout that - #newScript in both context and symbol is entirely obsolete and no longer used. You need to use #startScript: for everything, e.g.,
That does explain a couple of things. My confusion is that I was looking for a symmetric invocation to #newProcess and #fork. Part of the differences with scripts is that they are event driven. It would be useful to see how much code we could strip from your script framework that would still offer the event-loop behavior.
self startScript:
[Transcript cr; show: 'platformName: '; show: ((Smalltalk startScript: #platformName) waitUntil: #done; result) ]
Ok, this also does the disappearing script trick. I think it is due to a weakly held scriptProcess in the ScriptProcessLink. In browsing the regular ProcessorScheduler and Process, the links are not weak. It may also reject scheduling multiple scripts of the same selector, but I am not sure of that. The weakness of reference to suspended Scripts can be demonstrated with:
10 timesRepeat: [ self startScript: [ Transcript cr; show: 'platformName: '; show: (( Smalltalk startScript: #platformName) waitUntil: #done; result) ] ]
In another mail, Andreas wrote:
The other issue is that I think your #startScript: may start the new script before finishing the old one.
It will not start evaluating the "payload" of the script (the associated message) but it will start the associated process. This is needed in order to preserve the order of started scripts. Here's what happens: When you issue a #startScript: (with no trigger) a new script process will be initiated. The new process will run #privateRunMsg using a +1 priority of the default user script priority to preempt the current process. It will then call #startScriptProcess which schedules the user script with the processor scheduler. It is in ScriptScheduler>>scheduleScript: that this process is suspended. Only later, when the processor comes about this script again, it will give it control and allow the payload message to be evaluated.
I have a few questions. I don't think this is a good thing for the eventual event-loop. This is interrupting the current script and I don't think there are any guarantees that the currentScript will be the first script resumed after the new script is scheduled. Is there? There may be a third script which is scheduled ahead of the currentScript. How does the suspension of a script schedule the script for resumption? Is there no way to schedule the new script without making it the activeScript?
I need to change that to finish the current script before rescheduling any of the others. How would you suggest we do this?
I don't think you need or want this. The old script will always finish before the new one is run, unless it blocks. For example:
nil startScript:[ nil startScript:[ Transcript cr; show: 'inner'. ]. 1 to: 1000000 do:[:i| i*i]. Transcript cr; show: 'outer '. ].
Will always answer "outer" before "inner". Only if you change the outer script to block, the inner will be run before the outer is completed.
Ok, that works, but how is that guaranteed?
I will start to strip all of the cruft I have in SqueakElib that no longer applies now that the ScriptScheduler is managing the queuing of scripts. There is quite a bit of code I can get rid of. I will also consolidate to one hierarchy of sending contexts and shift the semantics of entering and leaving an eventual sending regime, to the selectors #eventual and #immediate, unless someone has some better semantics to suggest.
For the moment, I would like to include the ScriptProcessor as part of the Elib package, so that it can be loaded in a bare-bones squeak and we can focus on minimizing the code. Does this sound like a reasonable approach, Andreas?
cheers, rob
Rob,
The most important thing first:
Ok, that works, but how is that guaranteed?
Yes, yes, and yes. It is THE fundamental invariant. Scripts are only interrupted if they block. #startScript: is non-blocking. Period. What follows are mere details of how this is implemented ;-)
[Note: Read up to the end of the message before replying - various questions might be answered further down the line]
I have a few questions. I don't think this is a good thing for the eventual event-loop. This is interrupting the current script and I don't think there are any guarantees that the currentScript will be the first script resumed after the new script is scheduled. Is there?
As far as the "user script" (namely: the payload of the process) is concerned, #startScript: is immediate (non-blocking). It is the implementation of the synchronization semantics which will temporarily suspend the invoking user script but do so in a way that it is invisible from the client. The implementation essentially does:
* script1 issues #startScript: ===== system level ==== this is where the synchronization semantics starts. in order to preserve ordering of scripts the following happens: * #startScript: creates a new script (script2) with a higher(!) priority than script1 (Processor>>initialScriptPriority) * script2 preempts script1 * script2 schedules itself with the scheduler * script2 suspends itself (in scheduler's activeList) ===== user level ===== * script1 continues happily as if never interrupted [... some time later ...] * script2 gets its slot ===== system level ==== * script2 resets its priority down(!) to regular user script priority ===== user level ==== * script2 delivers payload
To get the scheduling semantics right, I am playing with process priorities. As long as scripts are "out in the wild" (that is: not controlled by the script processor) they run at higher priority. This happens only in two places: Immediately after their creation (in which case they need to schedule themselves) or inside "primitive waits". In both cases the priority is bumped up in order to ensure that there can be only a single script "out in the wild" and that this script reschedules itself as quickly as it can with the script scheduler. [Slight OT: There *is* a chance that two scripts are "in the wild" if both get released after waiting on a semaphore. However, in this case the user code cannot decide "which one was first" so ordering does not matter. For issuing #startScript: however, this will never happen, since #startScript: is only issued from scripts which aren't "out in the wild"].
There are many, many subtleties in this regime. I went to great length to make this work "just right".
There may be a third script which is scheduled ahead of the currentScript.
No. There is no way that this can happen. Literally. The scheduler process even runs at a lower priority than the script processes (once more: priorities are our friends ;) in order to ensure that a new script is only scheduled if the current script is blocking or completed. Using priorities ensures this invariant primitively and avoids needless synchronization of script/processor.
How does the suspension of a script schedule the script for resumption?
The script reschedules itself. When a script blocks, it always waits on some semaphore (this is the only way any process in Squeak can block). When this happens, the hooks within #beginWaitOn: and #endWaitOn: kick in. They provide the appropriate rescheduling semantics by bumping the priority up for the wait (the script is now "in the wild" since we don't know when it will become resumed primitively) and rescheduling itself (and reducing the priority) once it wakes up again.
One exception: Scripts inside critical sections *must* be run in the wild. The reason being that we would deadlock otherwise (I tried it ;)
Is there no way to schedule the new script without making it the activeScript?
You never schedule a new script and "automatically" make it the active script. When you schedule a script then (as the name "scheduling" says) its get added to the list of scripts that may be run as soon as the current one completes. Or do you mean "without adding it to the activeList"?!
Cheers, - Andreas
Andreas,
Thank you for the explanation. I was wondering a few things.
why do scripts have to run in the wild? you provide an initial moment when the script is running to initialize itself and schedule. Is it needed? why not run your own scheduling script at a higher priority? It would grab any newScript requests and schedule them..resuming the suspended script.
what am I missing?
cheers, rob
On Sunday, February 16, 2003, at 05:00 PM, Andreas Raab wrote:
Rob,
The most important thing first:
Ok, that works, but how is that guaranteed?
Yes, yes, and yes. It is THE fundamental invariant. Scripts are only interrupted if they block. #startScript: is non-blocking. Period. What follows are mere details of how this is implemented ;-)
[Note: Read up to the end of the message before replying - various questions might be answered further down the line]
I have a few questions. I don't think this is a good thing for the eventual event-loop. This is interrupting the current script and I don't think there are any guarantees that the currentScript will be the first script resumed after the new script is scheduled. Is there?
As far as the "user script" (namely: the payload of the process) is concerned, #startScript: is immediate (non-blocking). It is the implementation of the synchronization semantics which will temporarily suspend the invoking user script but do so in a way that it is invisible from the client. The implementation essentially does:
- script1 issues #startScript:
===== system level ==== this is where the synchronization semantics starts. in order to preserve ordering of scripts the following happens:
- #startScript: creates a new script (script2) with a higher(!)
priority than script1 (Processor>>initialScriptPriority)
- script2 preempts script1
- script2 schedules itself with the scheduler
- script2 suspends itself (in scheduler's activeList)
===== user level =====
- script1 continues happily as if never interrupted
[... some time later ...]
- script2 gets its slot
===== system level ====
- script2 resets its priority down(!) to regular user script priority
===== user level ====
- script2 delivers payload
To get the scheduling semantics right, I am playing with process priorities. As long as scripts are "out in the wild" (that is: not controlled by the script processor) they run at higher priority. This happens only in two places: Immediately after their creation (in which case they need to schedule themselves) or inside "primitive waits". In both cases the priority is bumped up in order to ensure that there can be only a single script "out in the wild" and that this script reschedules itself as quickly as it can with the script scheduler. [Slight OT: There *is* a chance that two scripts are "in the wild" if both get released after waiting on a semaphore. However, in this case the user code cannot decide "which one was first" so ordering does not matter. For issuing #startScript: however, this will never happen, since #startScript: is only issued from scripts which aren't "out in the wild"].
There are many, many subtleties in this regime. I went to great length to make this work "just right".
There may be a third script which is scheduled ahead of the currentScript.
No. There is no way that this can happen. Literally. The scheduler process even runs at a lower priority than the script processes (once more: priorities are our friends ;) in order to ensure that a new script is only scheduled if the current script is blocking or completed. Using priorities ensures this invariant primitively and avoids needless synchronization of script/processor.
How does the suspension of a script schedule the script for resumption?
The script reschedules itself. When a script blocks, it always waits on some semaphore (this is the only way any process in Squeak can block). When this happens, the hooks within #beginWaitOn: and #endWaitOn: kick in. They provide the appropriate rescheduling semantics by bumping the priority up for the wait (the script is now "in the wild" since we don't know when it will become resumed primitively) and rescheduling itself (and reducing the priority) once it wakes up again.
One exception: Scripts inside critical sections *must* be run in the wild. The reason being that we would deadlock otherwise (I tried it ;)
Is there no way to schedule the new script without making it the activeScript?
You never schedule a new script and "automatically" make it the active script. When you schedule a script then (as the name "scheduling" says) its get added to the list of scripts that may be run as soon as the current one completes. Or do you mean "without adding it to the activeList"?!
Cheers,
- Andreas
Squeak-e mailing list Squeak-e@lists.squeakfoundation.org http://lists.squeakfoundation.org/listinfo/squeak-e
what am I missing?
Simplicity, efficiency and robustness. And a guy named Heisenberg ;) Try it. It's the simplest solution since all of the critical invariants are ensured primitively. Just think about the scheduler itself - if it were running at a higher priority you'd need to tell it when to release the next process. But what if 'yer aborted, terminated, killed? How could the scheduler decide if there's already a script running? etc. You'd have to communicate too many things explicitly that I know for given implictly by the very fact some process is currently running.
Cheers, - Andreas
-----Original Message----- From: squeak-e-bounces@lists.squeakfoundation.org [mailto:squeak-e-bounces@lists.squeakfoundation.org] On Behalf Of Robert Withers Sent: Monday, February 17, 2003 6:23 AM To: Squeak-E - a capability-secure Squeak Subject: Re: [Squeak-e] adding eventual sending to Croquet'sScriptProcessandScriptScheduler
Andreas,
Thank you for the explanation. I was wondering a few things.
why do scripts have to run in the wild? you provide an initial moment when the script is running to initialize itself and schedule. Is it needed? why not run your own scheduling script at a higher priority? It would grab any newScript requests and schedule them..resuming the suspended script.
what am I missing?
cheers, rob
On Sunday, February 16, 2003, at 05:00 PM, Andreas Raab wrote:
Rob,
The most important thing first:
Ok, that works, but how is that guaranteed?
Yes, yes, and yes. It is THE fundamental invariant. Scripts are only interrupted if they block. #startScript: is non-blocking.
Period. What
follows are mere details of how this is implemented ;-)
[Note: Read up to the end of the message before replying - various questions might be answered further down the line]
I have a few questions. I don't think this is a good thing for the eventual event-loop. This is interrupting the current script and I don't think there are any guarantees that the currentScript will be the first script resumed after the new script is scheduled. Is there?
As far as the "user script" (namely: the payload of the process) is concerned, #startScript: is immediate (non-blocking). It is the implementation of the synchronization semantics which will
temporarily
suspend the invoking user script but do so in a way that it is invisible from the client. The implementation essentially does:
- script1 issues #startScript:
===== system level ==== this is where the synchronization semantics starts. in order to preserve ordering of scripts the following happens:
- #startScript: creates a new script (script2) with a higher(!)
priority than script1 (Processor>>initialScriptPriority)
- script2 preempts script1
- script2 schedules itself with the scheduler
- script2 suspends itself (in scheduler's activeList)
===== user level =====
- script1 continues happily as if never interrupted
[... some time later ...]
- script2 gets its slot
===== system level ====
- script2 resets its priority down(!) to regular user
script priority
===== user level ====
- script2 delivers payload
To get the scheduling semantics right, I am playing with process priorities. As long as scripts are "out in the wild" (that is: not
controlled by
the script processor) they run at higher priority. This happens
only in two
places: Immediately after their creation (in which case they need to schedule themselves) or inside "primitive waits". In both cases the priority is bumped up in order to ensure that there can be only a
single script
"out in the wild" and that this script reschedules itself as
quickly as it
can with the script scheduler. [Slight OT: There *is* a chance that two scripts are "in the wild" if both get released after waiting on a semaphore. However, in this case the user code cannot decide "which one was first" so ordering does not matter. For issuing #startScript:
however, this will
never happen, since #startScript: is only issued from scripts
which aren't
"out in the wild"].
There are many, many subtleties in this regime. I went to
great length
to make this work "just right".
There may be a third script which is scheduled ahead of the currentScript.
No. There is no way that this can happen. Literally. The scheduler process even runs at a lower priority than the script processes (once more: priorities are our friends ;) in order to ensure that a new
script is
only scheduled if the current script is blocking or completed. Using priorities ensures this invariant primitively and avoids needless
synchronization
of script/processor.
How does the suspension of a script schedule the script for resumption?
The script reschedules itself. When a script blocks, it
always waits
on some semaphore (this is the only way any process in Squeak can
block). When
this happens, the hooks within #beginWaitOn: and #endWaitOn:
kick in. They
provide the appropriate rescheduling semantics by bumping
the priority
up for the wait (the script is now "in the wild" since we
don't know when
it will become resumed primitively) and rescheduling itself
(and reducing
the priority) once it wakes up again.
One exception: Scripts inside critical sections *must* be
run in the
wild. The reason being that we would deadlock otherwise (I tried it ;)
Is there no way to schedule the new script without making it the activeScript?
You never schedule a new script and "automatically" make it
the active
script. When you schedule a script then (as the name "scheduling" says) its get added to the list of scripts that may be run as soon as the current one completes. Or do you mean "without adding it to the activeList"?!
Cheers,
- Andreas
Squeak-e mailing list Squeak-e@lists.squeakfoundation.org http://lists.squeakfoundation.org/listinfo/squeak-e
Squeak-e mailing list Squeak-e@lists.squeakfoundation.org http://lists.squeakfoundation.org/listinfo/squeak-e
On Monday, February 17, 2003, at 04:50 AM, Andreas Raab wrote:
what am I missing?
Simplicity, efficiency and robustness. And a guy named Heisenberg ;) Try it. It's the simplest solution since all of the critical invariants are ensured primitively. Just think about the scheduler itself - if it were running at a higher priority you'd need to tell it when to release the next process. But what if 'yer aborted, terminated, killed? How could the scheduler decide if there's already a script running? etc. You'd have to communicate too many things explicitly that I know for given implictly by the very fact some process is currently running.
And I clearly don't know enough yet to really come to a conclusion. I like Heisenberg. The inability to precisely measure position and velocity or time and energy hopefully have something to offer us in the world of asynchronous messaging. It would provide a little leeway in exactly when and where an event occurred and was handled. It's probably an emergent property of deadline based scheduling.
I am not sure why you couldn't attach a state machine to the process so that it would do the right thing when a scheduler Process asked it to reschedule itself. There looks to be quite a bit of state in ScriptProcess already. Since all scripts are at the same priority, you are allowed to do this, right? If they could have different priorities, then you could be interrupted. What is the low-priority scheduler Process? Is it the ticker?
I am currently more concerned with the lost scripts that I am experiencing. I opened a Process Browser and sure enough, there were all my Scripts waiting on Semaphores. I must have opened up a hole somewhere by a) submitting multiple scripts (in a loop) and/or b) not signaling a semaphore (or calling some method which does this under the covers).
I should have the next consolidated package this week sometime. As we discussed, I will include the Scripting changesets, so that it can be loaded separately from Croquet.
cheers, rob
Andreas, I wrote about some issues I was having with dropped or locked scripts. I need to debug this situation. How do I turn on traceEvents?
thanks, rob
On Monday, February 17, 2003, at 08:41 AM, Robert Withers wrote:
I am currently more concerned with the lost scripts that I am experiencing. I opened a Process Browser and sure enough, there were all my Scripts waiting on Semaphores. I must have opened up a hole somewhere by a) submitting multiple scripts (in a loop) and/or b) not signaling a semaphore (or calling some method which does this under the covers).
Rob,
I am not sure why you couldn't attach a state machine to the process so that it would do the right thing when a scheduler Process asked it to reschedule itself.
You could. It would simply be a different approach, one which requires much more care in a variety of places. For example, consider an error which throws up a debugger. You would have to take much care to handle this case correctly. Even worse, external interrupts (such as user interrupts) can interfere with your careful scheduling semantics. In every single case where the scheduler isn't explicitly notified your system will be entirely hosed with no way out. Because debugging this stuff is incredibly hard and making mistakes is easy, I wanted to have a way that is simply fault-tolerant in the vast majority of cases. After thinking about all the associated issues it seemed to me that making it so that the system "runs by default" is the best choice to make.
There looks to be quite a bit of state in ScriptProcess already. Since all scripts are at the same priority, you are allowed to do this, right?
Yes. And because of the scheduler semantics. This ensures that as long as you stay in the "script universe" everything works out right.
If they could have different priorities, then you could be interrupted.
But they don't. "Process priorities" are not part of the scripting universe.
What is the low-priority scheduler Process? Is it the ticker?
Yes, it's essentially the process which invokes #runActiveScripts and controls the "life line" of the system. In E terms it's the process which controls the VAT turn (I think).
I am currently more concerned with the lost scripts that I am experiencing. I opened a Process Browser and sure enough, there were all my Scripts waiting on Semaphores. I must have opened up a hole somewhere by a) submitting multiple scripts (in a loop) and/or b) not signaling a semaphore (or calling some method which does this under the covers).
Hard to tell. One note though: You should not assume things to work correctly from outside the scripting universe. IOW, when you do something that requires "script scheduling semantics" you cannot do this from a regular workspace *unless* you wrap it into a startScript: first. I have not provided the means to drive all of Morphic from the scripting regime.
Andreas, I wrote about some issues I was having with dropped or locked scripts. I need to debug this situation. How do I turn on traceEvents?
ScriptEventTracer trace: #eventName in: eventSignaler.
This will hand you an event tracer which is notified about *all* of the events and messages which directly or indirectly occur in response to the initial event (that is even nested signals from scripts and their responses will be reported). This can be a tree of events/messages (if there are multiple handlers of the event) and it can take forever (quite literally so if one of the scripts blocks forever). You can get the "current" (e.g., partial) list of responses by looking at its log (which contains ScriptEventTracerEntries).
Note: Part of the interface is missing (I just noticed this). The event tracer doesn't expose an interface for querying about its results.
Note^2: If you use the tracer and read out its contents you should do this from scripts - it does not provide any locks (the scripting universe has no need for it) and unexpectedly preempting it might turn out problematic.
Cheers, - Andreas
squeak-e@lists.squeakfoundation.org
-
Andreas Raab -
Robert Withers -
Withers, Robert