While running the tests (squeak 4.6) on macos32x86, the failure materializes in console with repeated:
select: Invalid argument errno 22
I cannot reproduce on my own mac... workaround: allow failure for this brand, it's a legacy VM, not a showstopper.
The sole instance of `perror("select");` is found in platforms/unix/vm/aio.c (other in minheadless and RiscOS do not count)
That's inside `aioPoll(long microSeconds)` What could possibly happen is an invalid timeval... Either a negative microSeconds or a greater than maximum possible timeout (implementation defined). The later cannot happen, because microSeconds is a long, and since this is a 32bits VM, the maximum value we can pass is about 2e9 microSeconds (a bit more than 2000 seconds), far from the limit (select man pages tells that the limit is at least 31 days, more than 2e6 seconds!). Inside the loop, there is a protection against `microSeconds <= 0`, so the only possibility for an invalid timeval is calling `aioPoll` with a negative timeout.
Maybe we should inquire if there is not an edge case in cog.v3 delay handling... It might be possible that `primitiveSignalAtMilliseconds(void) ` ask for a small delay in the past, leading to a negative `deltaMsecs` that will be wrapped to positive (modulo `MillisecondClockMask` which is a 29 bits mask), which multiplied by 1000, will lead to about 2^39 microseconds `nextWakeupUsecs` increment. For example, 1ms in the past will lead to a negative delay: when wrapped on 32bits long:
(16r1FFFFFFF-1*1000 \ (1 << 32-1)) highBit -> 32. 16r1FFFFFFF-1*1000 \ (1 << 32-1) - (1<<32) -> -1876.
Then `ioRelinquishProcessorForMicroseconds` will invoke `aioSleepForUsecs(realTimeToWait)` with this negative `long realTimeToWait`.
That's only a theory that came to my mind by reading code, this has to be confirmed, but it seems to me that we already solved that kind of flaw in the past... Maybe not for cog.v3?
So, there was a thread from March 2017 explaining the pitfalls in `primitiveSignalAtMilliseconds`:
Re: [Vm-dev] We need help from VM experts. Re: Freeze after Morph Activity
http://lists.squeakfoundation.org/pipermail/vm-dev/2017-March/024381.html
In particular, all was said here: the rollover workaround is **HIGHLY suspiscious**: http://lists.squeakfoundation.org/pipermail/vm-dev/2017-March/024410.html
So we solved it, or IMO we didn't solve, just patched: http://lists.squeakfoundation.org/pipermail/vm-dev/2017-March/024438.html
As I later reminded, there were still problems: http://lists.squeakfoundation.org/pipermail/vm-dev/2017-May/025160.html
But our attention was focused on more important developments (spur,win64 etc...) and we patched and replaced `usqLong deltaMsecs` by `sqLong deltaMsecs`, about 1 year after 1st problem: http://lists.squeakfoundation.org/pipermail/vm-dev/2018-March/027041.html
The compiler warning disappeared, but we kept the rollover that I once suggested to remove! We eliminate all smells of the problem, but not really the problem itself, that remain burried and quiet for some times (that's expected, squeak.cog.v3 might be used occasionally or less). I'm ready to bet a few hundred $ that we can can still trigger a problem in cog.v3 as Juan did once, and that our CI failures are related. I haven't got an old image handy near this keyboard, but if anyone wants to try this in a squeak.cog.v3, and watchout what the console delivers:
s := Semaphore new. Delay primSignal: s atMilliseconds: Time primMillisecondClock - 2. s wait.
Hehe, I couldn't resist and tried the snippet on another machine with available Squeak-4.6.image and above snippet just triggers the infinite
select: Invalid argument errno 22 select: Invalid argument errno 22 select: Invalid argument errno 22
So I know the origin (abusive Rollover protection) and I know how to fix. I just wonder if it is that much useful to spend time on legacy VM...
Fix is on its way, first stage at: http://source.squeak.org/VMMaker/VMMaker.oscog-nice.2572.diff
Wait, bad fix, we don't get errno 22 console litany in case of expired delay, but the image hang forever (the Semaphore never get signalled) http://source.squeak.org/VMMaker/VMMaker.oscog-nice.2573.diff
Ah, I see that the roll-over case was handled at image side already, at the end of `handleTimerEvent`
...snip... "Since we have processed all outstanding requests, reset the timing semaphore so that only new work will wake us up again. Do this RIGHT BEFORE setting the next wakeup call from the VM because it is only signaled once so we mustn't miss it." TimingSemaphore initSignals. Delay primSignal: TimingSemaphore atMilliseconds: nextTick.
"This last test is necessary for the obscure case that the msecs clock rolls over after nowTick has been computed (unlikely but not impossible). In this case we'd wait for MillisecondClockMask msecs (roughly six days) or until another delay gets scheduled (which may not be any time soon). In any case, since handling the condition is easy, let's just deal with it" Time millisecondClockValue < nowTick ifTrue:[TimingSemaphore signal]. "retry"
I added the protection to prevent `long` overflow in `ioRelinquishProcessorForMicroseconds`. The protection is not strictly necessary after https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/b80dd63c2aa1517b186.... However, I see that the `SuspendedDelays` sorting by `resumptionTime` will be BAD after a roll-over. Those greater than limit/2 should come before those less than limit/2! This should be handled at the end of `handleTimerEvent`. These delays might never get signalled...
This does not explain why CI still fails...
Bah, I think I can explain the failures now: http://source.squeak.org/VMMaker/VMMaker.oscog-nice.2574.diff
I've been blind enough to think in higher level Smalltalk rather than think in lower level C...
Thank you, Nicolas!
Closed #436.
vm-dev@lists.squeakfoundation.org
-
Nicolas Cellier -
Tobias Pape