Dear Eliot,
I tried to understand the basic load of the VM and I have two questions for the heartbeat code. Is it right that there are two concerns for the code?
1.) Call forceInterruptCheckFromHeartbeat periodically
2.) Update the clock as seen by the image
1st is useful to interrupt Processes like […] repeat with a pre-emptive scheduler? On a different note do backwards jumps generate interrupt checks too?
2nd is this an optimization to not have to generate the clock when the primitive is being used?
thanks holger
Hi Holger,
On Sep 13, 2015, at 10:37 AM, Holger Freyther holger@freyther.de wrote:
Dear Eliot,
I tried to understand the basic load of the VM and I have two questions for the heartbeat code. Is it right that there are two concerns for the code?
1.) Call forceInterruptCheckFromHeartbeat periodically
2.) Update the clock as seen by the image
1st is useful to interrupt Processes like […] repeat with a pre-emptive scheduler? On a different note do backwards jumps generate interrupt checks too?
Yes
2nd is this an optimization to not have to generate the clock when the primitive is being used?
Nearly.
You are correct for #1. The main use is 1. To cause a breakout from machine code at regular intervals (default every 2ms). Backward jumps also perform the test. Using counters instead does not work well; counters are expensive to update in every send and the frequency of breakout varies wildly depending on whether one is calling code containing slow primitives (eg LargeIntegerArithmetic) or not.
For 2, getting the heartbeat to update the clock is for delay expiry. Fetching the time is slow, and delays need only have about a millisecond resolution. So having the heartbeat update the clock saves time when seeing if the active delay has fired. At first it was used also to update the time seen through the time primitives but Dave had me change it. This gives you much better (microsecond) clock resolution at the cost of a slower primitive. That's probably a good default choice. It does lessen the case for doing this at all though. That's an experiment that I'm interested in the results of.
There is another use of the heartbeat in the threaded FFI. When a call out releases the VM the heartbeat is used to detect that the VM is free and that another thread should attempt to run it.
Sent from my iPhone
thanks holger
On 13 Sep 2015, at 19:53, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Holger,
Hi!
thank you for the super quick reply!
You are correct for #1. The main use is 1. To cause a breakout from machine code at regular intervals (default every 2ms). Backward jumps also perform the test. Using counters instead does not work well; counters are expensive to update in every send and the frequency of breakout varies wildly depending on whether one is calling code containing slow primitives (eg LargeIntegerArithmetic) or not.
For 2, getting the heartbeat to update the clock is for delay expiry. Fetching the time is slow, and delays need only have about a millisecond resolution. So having the heartbeat update the clock saves time when seeing if the active delay has fired. At first it was used also to update the time seen through the time primitives but Dave had me change it. This gives you much better (microsecond) clock resolution at the cost of a slower primitive. That's probably a good default choice. It does lessen the case for doing this at all though. That's an experiment that I'm interested in the results of.
can you think of a strategy to stop the timer when nothing is running inside the image? I see getNextWakeupUsecs in ioRelinquishProcessorForMicroseconds but that does’t seem to disable the interval timer? E.g. if I look at the mac vm and ioRelinquishProcessorForMicroseconds it doesn’t appear to stop the itimer?
I am looking at this from the aioPoll/SIGIO point of view. I was surprised how close we are to have a system that doesn’t consume CPU when nothing is running. Maybe there is something I am missing (I don’t know win32) but if the IdleProcess starts to run the system could:
* Stop the heartbeat timer * Have the mac variant of ioRelinquishProcessorForMicroseconds for unix and get the time the next Delay expires. * Set the heartbeat to expire at this time? * Yield execution and wait for the SIGIO (or the timer)? * On return to the image call aioPoll if the SIGIO occurred?
Is there any general objection to this?
kind regards holger
I looked at this six years back and altered the code base to pause the heart beat thread until we know when we have to wakeup .
Results were inclusive, so more testing is welcome.
(a) how much overhead is there to pause and wakeup the thread? If you sleep for 2 ms, is it worth it? (b) because of the morphic polling look we are woken every 16-20 milliseconds. (c) There is a race? between what the VM thinks the next wakeup time, and what the ioRelinquishProcessorForMicroseconds is expected. Sometimes you are asked to sleep, yet the internal VM timer says you should wakeup to service a task right now or a millisecond in the past. Yet the VM had decided there was no runable tasks just microseconds ago.
(d) Do mouse/trackpad/keyboard/socket/FD events actually awake the paused program?
On Sun, Sep 13, 2015 at 1:00 PM, Holger Freyther holger@freyther.de wrote:
On 13 Sep 2015, at 19:53, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Holger,
Hi!
thank you for the super quick reply!
You are correct for #1. The main use is 1. To cause a breakout from
machine code at regular intervals (default every 2ms). Backward jumps also perform the test. Using counters instead does not work well; counters are expensive to update in every send and the frequency of breakout varies wildly depending on whether one is calling code containing slow primitives (eg LargeIntegerArithmetic) or not.
For 2, getting the heartbeat to update the clock is for delay expiry.
Fetching the time is slow, and delays need only have about a millisecond resolution. So having the heartbeat update the clock saves time when seeing if the active delay has fired. At first it was used also to update the time seen through the time primitives but Dave had me change it. This gives you much better (microsecond) clock resolution at the cost of a slower primitive. That's probably a good default choice. It does lessen the case for doing this at all though. That's an experiment that I'm interested in the results of.
can you think of a strategy to stop the timer when nothing is running inside the image? I see getNextWakeupUsecs in ioRelinquishProcessorForMicroseconds but that does’t seem to disable the interval timer? E.g. if I look at the mac vm and ioRelinquishProcessorForMicroseconds it doesn’t appear to stop the itimer?
I am looking at this from the aioPoll/SIGIO point of view. I was surprised how close we are to have a system that doesn’t consume CPU when nothing is running. Maybe there is something I am missing (I don’t know win32) but if the IdleProcess starts to run the system could:
* Stop the heartbeat timer * Have the mac variant of ioRelinquishProcessorForMicroseconds forunix and get the time the next Delay expires. * Set the heartbeat to expire at this time? * Yield execution and wait for the SIGIO (or the timer)? * On return to the image call aioPoll if the SIGIO occurred?
Is there any general objection to this?
kind regards holger
On 13 Sep 2015, at 22:11, John McIntosh johnmci@smalltalkconsulting.com wrote:
Hi!
I looked at this six years back and altered the code base to pause the heart beat thread until we know when we have to wakeup .
Results were inclusive, so more testing is welcome.
(a) how much overhead is there to pause and wakeup the thread? If you sleep for 2 ms, is it worth it? (b) because of the morphic polling look we are woken every 16-20 milliseconds. (c) There is a race? between what the VM thinks the next wakeup time, and what the ioRelinquishProcessorForMicroseconds is expected. Sometimes you are asked to sleep, yet the internal VM timer says you should wakeup to service a task right now or a millisecond in the past. Yet the VM had decided there was no runable tasks just microseconds ago.
Thank you for the insight, so depends on the usecase. I don’t know Morphic/WorldState enough to judge if the polling/delay loop is necessary. The usecase I look at is for an image that is mostly waiting for socket i/o (e.g. incoming REST call, TCP command) and the “idle” load is kind of a issue (running 10 images might already fill one CPU core).
Let’s hope i find enough time over the next couple of weeks to work on it for the Unix VM
(d) Do mouse/trackpad/keyboard/socket/FD events actually awake the paused program?
For the X11 code SIGIO is enabled for the XDisplay connection so an event would lead to a wake-up.
Hi Holger,
On Sun, Sep 13, 2015 at 1:00 PM, Holger Freyther holger@freyther.de wrote:
On 13 Sep 2015, at 19:53, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Holger,
Hi!
thank you for the super quick reply!
You are correct for #1. The main use is 1. To cause a breakout from
machine code at regular intervals (default every 2ms). Backward jumps also perform the test. Using counters instead does not work well; counters are expensive to update in every send and the frequency of breakout varies wildly depending on whether one is calling code containing slow primitives (eg LargeIntegerArithmetic) or not.
For 2, getting the heartbeat to update the clock is for delay expiry.
Fetching the time is slow, and delays need only have about a millisecond resolution. So having the heartbeat update the clock saves time when seeing if the active delay has fired. At first it was used also to update the time seen through the time primitives but Dave had me change it. This gives you much better (microsecond) clock resolution at the cost of a slower primitive. That's probably a good default choice. It does lessen the case for doing this at all though. That's an experiment that I'm interested in the results of.
can you think of a strategy to stop the timer when nothing is running inside the image? I see getNextWakeupUsecs in ioRelinquishProcessorForMicroseconds but that does’t seem to disable the interval timer? E.g. if I look at the mac vm and ioRelinquishProcessorForMicroseconds it doesn’t appear to stop the itimer?
You've read what John has said. I think the easiest thing to do is not to stop and start the heartbeat thread, but merely to set a flag when entering ioRelinquishProcessorForMicroseconds, clearing it on exit, and have the heartbeat test this flag before doing anything that could disturb the VM thread(s). The overhead of the heartbeat thread is very low. It doesn't call select or anything like that. All it does is spin, calling nanosleep, polling the clock and then setting a variable. For example, avoiding polling the clock when in ioRelinquishProcessorForMicroseconds might save a few cycles.
I am looking at this from the aioPoll/SIGIO point of view. I was surprised how close we are to have a system that doesn’t consume CPU when nothing is running. Maybe there is something I am missing (I don’t know win32) but if the IdleProcess starts to run the system could:
* Stop the heartbeat timer
or set the flag that gets it to do nothing. It may be more efficient to set this flag an have it tested than stop and start the heartbeat thread.
* Have the mac variant of ioRelinquishProcessorForMicroseconds forunix and get the time the next Delay expires. * Set the heartbeat to expire at this time?
Make sure that the blocking call to poll or whatever will time out at or shortly before the next delay expiry.
* Yield execution and wait for the SIGIO (or the timer)?
* On return to the image call aioPoll if the SIGIO occurred?
I think blocking in epoll and the like is the right approach. That gives control to the kernel for the most efficient possible wait state.
Is there any general objection to this?
Absolutely not. Quite the opposite. It would be great to see the VM block efficiently. Great to have your energy!! Thank you!
kind regards holger
_,,,^..^,,,_ best, Eliot
Hi Holger,
I don't want to put you off the path of improving things, but did you see the socket plugin patch that lets you start an image from a socket event using systemd?
see: http://forum.world.st/DEV-Systemd-socket-activation-patch-for-the-VM-td47670...
Holger Freyther wrote
On 13 Sep 2015, at 22:11, John McIntosh <
johnmci@
> wrote:
Hi!
I looked at this six years back and altered the code base to pause the heart beat thread until we know when we have to wakeup .
Results were inclusive, so more testing is welcome.
(a) how much overhead is there to pause and wakeup the thread? If you sleep for 2 ms, is it worth it? (b) because of the morphic polling look we are woken every 16-20 milliseconds. (c) There is a race? between what the VM thinks the next wakeup time, and what the ioRelinquishProcessorForMicroseconds is expected. Sometimes you are asked to sleep, yet the internal VM timer says you should wakeup to service a task right now or a millisecond in the past. Yet the VM had decided there was no runable tasks just microseconds ago.
Thank you for the insight, so depends on the usecase. I don’t know Morphic/WorldState enough to judge if the polling/delay loop is necessary. The usecase I look at is for an image that is mostly waiting for socket i/o (e.g. incoming REST call, TCP command) and the “idle” load is kind of a issue (running 10 images might already fill one CPU core).
Let’s hope i find enough time over the next couple of weeks to work on it for the Unix VM
(d) Do mouse/trackpad/keyboard/socket/FD events actually awake the paused program?
For the X11 code SIGIO is enabled for the XDisplay connection so an event would lead to a wake-up.
-- View this message in context: http://forum.world.st/Tasks-of-the-heartbeat-code-tp4849879p4850190.html Sent from the Squeak VM mailing list archive at Nabble.com.
On 14 Sep 2015, at 22:43, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Holger,
Dear Eliot,
time flies but I finally made a bit of progress.
You've read what John has said. I think the easiest thing to do is not to stop and start the heartbeat thread, but merely to set a flag when entering ioRelinquishProcessorForMicroseconds, clearing it on exit, and have the heartbeat test this flag before doing anything that could disturb the VM thread(s). The overhead of the heartbeat thread is very low. It doesn't call select or anything like that. All it does is spin, calling nanosleep, polling the clock and then setting a variable. For example, avoiding polling the clock when in ioRelinquishProcessorForMicroseconds might save a few cycles.
I have done my current experiments in uild.linux32x86/squeak.cog.v3/build.itimerheartbeat. I don't know if that is the closest that the pharovm is using but the issue is that it creates work when the system would be idle and as such "cost" (either your cloud platform charges you for CPU usage, or it takes resources from your cluster, etc).
I have done it using the X11 plugin:
1.) Don't disable SIGIO for the X11 handle:
@@ -6723,7 +6775,7 @@ aioHandle(browserPipes[0], npHandler, AIO_RX); } isConnectedToXServer= 1; - aioEnable(stXfd, 0, AIO_EXT); + aioEnable(stXfd, 0, 0); aioHandle(stXfd, xHandler, AIO_RX); } return 0;
2.) Hack (needs to use CAS or such) ioProcessEvents. So we remember when a SIGIO has been received and instead of just trying to poll every nextPollUsecs we poll if we know it will succeed.
--- ../../src/vm/gcc3x-cointerp.c (revision 3546) +++ ../../src/vm/gcc3x-cointerp.c (working copy) @@ -39583,6 +39583,7 @@ }
+int doDispatch = 0; /* Check for possible interrupts and handle one if necessary. Answer if a context switch has occurred. */
@@ -39661,15 +39662,13 @@ switched = 1; } } - if (((now = ioUTCMicroseconds())) >= GIV(nextPollUsecs)) { + + if (doDispatch) { + doDispatch = 0; GIV(statIOProcessEvents) += 1; ioProcessEvents(); - - /* msecs to wait before next call to ioProcessEvents. Note that strictly - speaking we might need to update 'now' at this point since - ioProcessEvents could take a very long time on some platforms */ - GIV(nextPollUsecs) = now + 20000; } + now = ioUTCMicroseconds();
if (GIV(interruptPending)) {
--- vm/aio.c (revision 3546) +++ vm/aio.c (working copy) @@ -143,10 +143,17 @@ #endif
/* initialise asynchronous i/o */ +void sigio_signal(int sig) +{ + extern void forceInterruptCheck(int); /* not really, but hey */ + extern int doDispatch; + doDispatch = 1; + forceInterruptCheck(sig); +}
+ void aioInit(void) { - extern void forceInterruptCheck(int); /* not really, but hey */
FD_ZERO(&fdMask); FD_ZERO(&rdMask); @@ -155,7 +162,7 @@ FD_ZERO(&xdMask); maxFd= 0; signal(SIGPIPE, SIG_IGN); - signal(SIGIO, forceInterruptCheck); + signal(SIGIO, sigio_signal); }
3.) When the relinquish primitive is called
--- vm-display-X11/sqUnixX11.c (revision 3546) +++ vm-display-X11/sqUnixX11.c (working copy) @@ -4828,8 +4828,38 @@
static sqInt display_ioRelinquishProcessorForMicroseconds(sqInt microSeconds) { - aioSleepForUsecs(handleEvents() ? 0 : microSeconds); + sigset_t set; + sigemptyset(&set); + + extern usqLong getNextWakeupUsecs(); + if (getNextWakeupUsecs() == 0) { + sigsuspend(&set); + } else { + usqLong nextWakeUp = getNextWakeupUsecs(); + usqLong now = ioUTCMicroseconds(); + struct itimerval value; + + sqLong deltaMicro = nextWakeUp - now; + if (deltaMicro < 1) + deltaMicro = 1; + /* don't re-run automatically */ + value.it_interval.tv_sec = value.it_interval.tv_usec = 0; + value.it_value.tv_sec = deltaMicro / 1000000; + value.it_value.tv_usec = deltaMicro % 1000000; + setitimer(ITIMER_REAL, &value, &value); + sigsuspend(&set); + value.it_value = value.it_interval; + setitimer(ITIMER_REAL, &value, NULL); + + extern int doDispatch; + if (doDispatch) { + doDispatch = 0; + aioPoll(0); + } + } + return 0; +#endif }
let me follow up with the "result" of my test.
holger
On 25 Dec 2015, at 21:40, Holger Freyther holger@freyther.de wrote:
I have done it using the X11 plugin:
1.) Don't disable SIGIO for the X11 handle: 2.) Hack (needs to use CAS or such) ioProcessEvents. So we remember when a SIGIO has been received and instead of just trying to poll every nextPollUsecs we poll if we know it will succeed.
I think these two changes make sense by itself. For platforms we have SIGIO we should enable it unconditionally and rely on it. The question is if we want to use atomic operations to have "doDispatch" change to 1 when it is '0' and atomically change it '1' to '0'?
3.) When the relinquish primitive is called
- extern usqLong getNextWakeupUsecs();
- if (getNextWakeupUsecs() == 0) {
- sigsuspend(&set);
This _never_ happens. It means that at least one part of the system is driven by delays. That is a bit unfortunate for network systems (e.g. unless I use RFB I do not need to have Morph update)
} else {
setitimer(ITIMER_REAL, &value, NULL);
This is something I don't understand. The interactive responsiveness is really terrible when I stop the heart beat timer. I assume that primitiveRelinquishProcessor will not lead to a context switch so it takes longer until a new task will be selected.
Eliot do you have the full picture between VM and Image here? In theory after doing 1st and 2nd the VM should now more quickly call ioProcessEvents and as it is not sleeping longer than the time to the next wake up time the interactive performance should not have changed.
kind regards holger
On 25 Dec 2015, at 22:03, Holger Freyther holger@freyther.de wrote:
This _never_ happens. It means that at least one part of the system is driven by delays. That is a bit unfortunate for network systems (e.g. unless I use RFB I do not need to have Morph update)
"And signal when the next request is due. We sleep at most 1sec here as a soft busy-loop so that we don't accidentally miss signals."
Both Pharo and Squeak seem to do that. This looks a bit odd and I wonder if anybody remembers if there have been issues with the wakeup in the past?
kind regards and I wish all of you a happy holiday season
holger
PS: With the Cogit/StackInterpreter (not spur) I seem to end up having the wrong receiver in top of stack. I don't know if that is a Pharo/Opal issue or something with the VM itself.
On 25 Dec 2015, at 21:40, Holger Freyther holger@freyther.de wrote:
Dear Eliot,
I was thinking what to do to make the below race free? I think I will:
* Block the SIGIO * Memory barrier? * Check if an event is already queued and then restore mask and return * Set itimer * sigsuspend * Unblock SIGIO * dispatch
Does this look correct? At the same time I think the per display backend relinquish code can be replaced with a generic unix one? Maybe even separating waiting for a timeout and waiting for I/O could be done and creates a better internal interface?
kind regards holger
static sqInt display_ioRelinquishProcessorForMicroseconds(sqInt microSeconds) {
- aioSleepForUsecs(handleEvents() ? 0 : microSeconds);
- sigset_t set;
- sigemptyset(&set);
- extern usqLong getNextWakeupUsecs();
- if (getNextWakeupUsecs() == 0) {
- sigsuspend(&set);
- } else {
- usqLong nextWakeUp = getNextWakeupUsecs();
- usqLong now = ioUTCMicroseconds();
- struct itimerval value;
- sqLong deltaMicro = nextWakeUp - now;
- if (deltaMicro < 1)
deltaMicro = 1;- /* don't re-run automatically */
- value.it_interval.tv_sec = value.it_interval.tv_usec = 0;
- value.it_value.tv_sec = deltaMicro / 1000000;
- value.it_value.tv_usec = deltaMicro % 1000000;
- setitimer(ITIMER_REAL, &value, &value);
- sigsuspend(&set);
- value.it_value = value.it_interval;
- setitimer(ITIMER_REAL, &value, NULL);
- extern int doDispatch;
- if (doDispatch) {
doDispatch = 0;aioPoll(0);- }
- }
On 25 Dec 2015, at 21:40, Holger Freyther holger@freyther.de wrote:
Dear Eliot,
- if (((now = ioUTCMicroseconds())) >= GIV(nextPollUsecs)) {
- if (doDispatch) {
doDispatch = 0;
/* msecs to wait before next call to ioProcessEvents. Note that strictlyspeaking we might need to update 'now' at this point sinceioProcessEvents could take a very long time on some platforms */GIV(nextPollUsecs) = now + 20000;
now = ioUTCMicroseconds();
if (GIV(interruptPending)) {
could you please help me with getting the above right and help me to get it into the VMMaker? I wonder which atomic interface to use between the aioPoll code and the VMMaker without breaking the Windows/OSX build right now.
1.) I think I don't want the time based polling but only if something has happened
2.) In the SIGIO signal handler I will need to register an event. I'm wondering if I want to use: __sync_fetch_and_add to count the number of SIGIOs not handled? otherwise the classic loop with __sync_bool_compare_and_swap?
3.) In the VMMaker and only if the SIGIO interface is present. I think I need to use sqCompareAndSwap to try to read the number of events?
4.) Which variables (and addresses) can be used and where would I keep them?
kind regards holger
On 01 Jan 2016, at 18:33, Holger Freyther holger@freyther.de wrote:
Good Evening Eliot, all,
I had to wait quite some time for my next flight and looked a bit at the more modern builtin atomic support.
- if (((now = ioUTCMicroseconds())) >= GIV(nextPollUsecs)) {
- if (doDispatch) {
doDispatch = 0;
/* msecs to wait before next call to ioProcessEvents. Note that strictlyspeaking we might need to update 'now' at this point sinceioProcessEvents could take a very long time on some platforms */GIV(nextPollUsecs) = now + 20000;
now = ioUTCMicroseconds();
if (GIV(interruptPending)) {
could you please help me with getting the above right and help me to get it into the VMMaker? I wonder which atomic interface to use between the aioPoll code and the VMMaker without breaking the Windows/OSX build right now.
1.) I think I don't want the time based polling but only if something has happened
I think this still holds true. If we miss a SIGIO event I want it to fail miserably instead of being lucky by a periodic fix-up. :)
2.) In the SIGIO signal handler I will need to register an event. I'm wondering if I want to use: __sync_fetch_and_add to count the number of SIGIOs not handled? otherwise the classic loop with __sync_bool_compare_and_swap?
Unless I am completely tired I think we are fine with an atomic store of '1'
void sigIO_handler(int sig) { __atomic_store_n(&doDispatch, 1, __ATOMIC_RELAXED) }
3.) In the VMMaker and only if the SIGIO interface is present. I think I need to use sqCompareAndSwap to try to read the number of events?
A simple exchange seems to do?
void check_event() { uint32_t should_dispatch; should_dispatch = __atomic_exchange_n(&doDispatch, 0, __ATOMIC_RELAXED); if (should_dispatch) ioProcessEvents(); }
The integration into VMMaker and display interface is probably going to be more involved. I will try to hack together a "current" idle vm with these primitives.
kind regards holger
vm-dev@lists.squeakfoundation.org
-
Eliot Miranda -
Holger Freyther -
John McIntosh -
Paul DeBruicker