On Mon, Mar 28, 2005 at 12:14:03AM -0800, John M McIntosh wrote:
> I'll comment on how the current implementation works, hah I hope I get  
> most of it correct. I know other on the list are actually responsible  
> for it's creation.
> If you load the vmmaker squeak map package you will find.

Thanks for the info.  A few points are still puzzling me, with the key
issue being how much trouble a low-priority process can make.

> 
> Interpreter>>transferTo: aProc
> 
> Is the method that actually transfer control between active process to  
> another, outside of direct control of any Smalltalk code.
> 
> This method is called via
> Interpreter>>primitiveSuspend
> Interpreter>>primitiveWait
> Interpreter>>primitiveYield
> Interpreter>>resume:
> 
> On Wait and Yield we add the current process to the end of a linked  
> list based on it's priority, then we wake the first runnable process  
> based on looking from highest to lowest priority.
> If there is no runnable process the VM will quit, with  'scheduler  
> could not find a runnable process', readers of the list are welcome to  
> read the list archives to find the change set I managed to get into the  
> update stream which led to that condition for the first few early  
> adopters one weekend many years ago.
> 
> For Resume: we put the active process to sleep and run the indicated  
> process if the indicated process has a higher priority.
> 
> Suspend is used mostly in the interface by the debugger and  
> ProcessBrower to control the runability of an indicated process.
> 
> This algorithm ensures:
> a) high priority processes will run first.
> b) all process at a given priority level will run in round robin  
> fashion if process switching occurs.
> 
> 
> When a process is resumed depends on what it is doing. It could be  
> waiting on a semaphore, or on a delay. Because of how resume: works a  
> process can become runnable but not run if a higher priority task can  
> run. This means of course a higher priority process that does not yield  
> control it will block runnable lower priority processes, or processes  
> at the same priority.

I thought from earlier remarks (not by you) that even a lower priority
process could block a higher priority process, if the former did not
yield control.

Obviously if you have a lower priority (smalltalk) process a higher
priority process can block you, but I thought the problems did not end
there.  Most of the comments in this email thread indicate that the
initial example of the email thread was a problem only because it was
running at a relatively high priority.  But some of the comments
indicate the problem is the processor (as in processor vs email, not
as in the class Processor) thread is not yielding control,
regardless of its priority.

> 
> But if for example you have a server application that accepts incoming  
> requests then forks off work at a lower priority this allows the   
> higher priority listeners to
> accept work, then lower priority tasks perform the actual work, but are  
> prevented from blocking higher priority process from accepting work. I  
> think someone earlier
> commented on forking work at the user background priority to prevent  
> the UI from locking up.
> 
> In general most usages of yield I have seen are in endless while loops  
> where one polls for work churning CPU cycles as a side effect and  

But below you say every method  send is an opportunity to yield, so
explicitly coded yields seem unnecessary.  More about this below.

> without the yield blocking any other runable processes. For the most  
> part usage of yield in the image follow this pattern.
> 
> "In Theory" there is nothing to prevent you from creating a processor  
> scheduler that wakes up every N milli-seconds and decides to suspend or  
> change the priority of the running process and picking a different  
> process to resume. There would be some question if this is 'safe',   
> however I do recall in VisualWorks at some point there was an optional  
> changeset to provide this functionality.  'Safe' here means when does a  
> process lose control and what does that mean for multi-threading. That  
> I can't answer, mind if someone could point to a Use case that would be  
> a plausible failure condition that would be interesting.
> 
> When processing switching can occur is of course a question to ask.  
> This occurs.
> when a method send is made or
> when a long unconditionalJump bytecode is executed in the backwards  
> direction. > 9 bytecodes?
> on a primitive call if  when the call takes > 1 or perhaps 17ms  
> depending on platform.
> 

The preceding list seems to indicate that explicitly coding a yield
would almost never be necessary.  Even the problematic case of a
callout to a long-lived db transaction, mentioned elsewhere in this
email thread as a potential problem, seems to be covered by the
process switching that occurs on a slow primitive call (your third
case above).  How the VM could get control back, in the absence of
using native threads (if only internally) is not clear to me in that
case.

So I'm still a bit confused.