On Thu, Oct 8, 2015 at 3:43 PM, Craig Latta <craig at netjam.org> wrote:

>
>
> > I guess the system is never arranging to unlock the Vm so there is
> > never any opportunity for the foreign callback to get entry.  Try
> > adding an occasional "Processor relinquishProcessorForMicroseconds:
> > 100" and see if that gives your call-back a window.
>
>      Well, my system is deadlocked in the foreign thread, forever trying
> and failing to own the VM when the library I'm using attempts to invoke
> my callback. I guess I have to do something more drastic, like kludge
> the unlock logic.
>
>      How should the system be arranging to unlock the VM?
>

Hmmm, the code is better than I thought.  So here's what it does:

ownVMFromUnidentifiedThread
...
count := 0. "If the current thread doesn't have an index it's new to the vm
and we need to allocate a new threadInfo, failing if we can't. We also need
a process in the foreignCallbackProcessSlot upon which to run the thread's
eventual callback." [[cogThreadManager tryLockVMToIndex: -1] whileFalse:
[self waitingPriorityIsAtLeast: foreignCallbackPriority. cogThreadManager
ioTransferTimeslice]. (objectMemory splObj: foreignCallbackProcessSlot) ~=
objectMemory nilObject] whileFalse: [cogThreadManager releaseVM. (count :=
count + 1) > 1000 ifTrue: [^-2]. cogThreadManager ioMilliSleep: 1].

and

waitingPriorityIsAtLeast: minPriority "Set the maxWaitingPriority to at
least minPriority on behalf of a thread wanting to acquire the VM. If
maxWaitingPriority is increased, schedule a thread activation check asap."
maxWaitingPriority < minPriority ifTrue: [maxWaitingPriority :=
minPriority. checkThreadActivation := true. self forceInterruptCheck]

So forceInterruptCheck will force the VM to break out of machine code, if
it is in machine code (this we need to check; for example it might be doing
relinquishProcessorForMicroseconds:). Once in
checkForEventsMayContextSwitch (the code that gets called to process
events) there is an attempt to attend to checkThreadActivation:

checkForEventsMayContextSwitch: mayContextSwitch "Check for possible
interrupts and handle one if necessary. Answer if a context switch has
occurred." ... switched := false. self assert: deferThreadSwitch not.
deferThreadSwitch := true. ... "inIOProcessEvents prevents reentrancy into
ioProcessEvents and allows disabling ioProcessEvents e.g. for native GUIs.
We would like to manage that here but can't since the platform code may
choose to call ioProcessEvents itself in various places." (now := self
ioUTCMicroseconds) >= nextPollUsecs ifTrue: [statIOProcessEvents :=
statIOProcessEvents + 1. self ioProcessEvents. "sets interruptPending if
interrupt key pressed; may callback" nextPollUsecs := now + 20000 "msecs to
wait before next call to ioProcessEvents. Note that strictly speaking we
might need to update 'now' at this point since ioProcessEvents could take a
very long time on some platforms"]. interruptPending ifTrue:
[interruptPending := false. "reset interrupt flag" sema := objectMemory
splObj: TheInterruptSemaphore. (sema ~= objectMemory nilObject and: [self
synchronousSignal: sema]) ifTrue: [switched := true]]. nextWakeupUsecs ~= 0
ifTrue: [now >= nextWakeupUsecs ifTrue: [nextWakeupUsecs := 0. "set timer
interrupt to 0 for 'no timer'" sema := objectMemory splObj:
TheTimerSemaphore. (sema ~= objectMemory nilObject and: [self
synchronousSignal: sema]) ifTrue: [switched := true]]]. ...
deferThreadSwitch := false. checkThreadActivation ifTrue:
[checkThreadActivation := false. self cedeToHigherPriorityThreads]. "N.B.
This may not return if we do switch." self threadSwitchIfNecessary: self
activeProcess from: CSCheckEvents. ^switched

So cedeToHigherPriorityThreads should do what it says. What does it do?

Well it does lots of clever things, but AFAICT it *doesn't* unlock the VM
if a higher-priority foreign callback is trying to own the VM. So that
would be the bug.

If you want to make progress on this you'll need to

a) build a simulator image (instructions on the blog)
b) set-up a test case in the simulator to exercise the
cedeToHigherPriorityThreads
logic and replicate the bug
c) have a go at rewriting cedeToHigherPriorityThreads and/or
threadSwitchIfNecessary:from:
to do the right thing.

Looks like the issue is at the bottom of threadSwitchIfNecessary:from: :

threadSwitchIfNecessary: newProc from: sourceCode
"Invoked from transferTo:from: to switch threads if the new process is
bound or affined to some other thread."
....
(self quickFetchInteger: PriorityIndex ofObject: newProc) <
maxWaitingPriority ifTrue:
[checkThreadActivation := true.
self forceInterruptCheck]

This will just spin causing the VM to do checkForEventsMayContextSwitch:.

Instead it probably needs to do something like

(self quickFetchInteger: PriorityIndex ofObject: newProc) <
maxWaitingPriority ifTrue:
[checkThreadActivation := true.
 self returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source:
source]

But this really needs to be thought through carefully.  It's a while since
I thought about this and I'm in no position to do so now.
Anyway, HTH

_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151008/154eca77/attachment.htm