<br><br><div class="gmail_quote">On Wed, Mar 3, 2010 at 9:29 PM, Andreas Raab <span dir="ltr"><<a href="mailto:andreas.raab@gmx.de">andreas.raab@gmx.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">On 3/3/2010 6:32 PM, Eliot Miranda wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
No, it is /broken/. What exception handlers are in effect when an<br>
unwind is run from another process? Its not just tat process identity<br>
isn't being preserved (which I would argue is important but not<br>
essential) its that unwinds are being run in an entirely arbitrary<br>
context which has nothing to do with their running normally. e.g.<br>
<br>
givethAndTakethAway: aCollection<br>
[[aCollection do: [:k| mydict at: k put: self thing].<br>
self doStuff] ensure: [aCollection do: [:k| mydict removeKey: k]]<br>
on: KeyNotFoundError<br>
do: [:ex| ex proceedWith: nil]<br>
<br>
will remove whatever keys in mydict are in aCollection when run<br>
normally, but when run if terminated from another process will break if<br>
any keys in aCollection have already been removed from mydict.<br>
</blockquote>
<br></div>
Yes, but my claim was that this a feature, not a bug, for a value of "feature" which means that the terminator (<- I really wanted to use that :-) is capable of controlling the exception context of the ensure block.<div class="im">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Running unwinds in another process is /broken/, period.<br>
</blockquote>
<br></div>
The reason why I'm not convinced of that, period or not, repetitions or not, is because we've seen no evidence of it ever being a problem. If it's not a problem, neither in our production environments that have triggered all the problems and the fixes for process, semaphore, mutex, termination etc. handling over the last years, nor in messages or bug reports that I've seen on this or other lists, I'm just not buying the strong claim you are making.<br>
<br>
Where is the *evidence* to support your claim? Not the theoretical discussion of what could possibly go wrong, but rather the actual, day-to-day things that people really get bitten by.</blockquote><div><br></div><div>Evidence will be hard to come by. In Squeak people will have coded around the limitation. I don't have access to the server applications in VW which might or might not be affected by this. But I can say that VW implements unwinds on termination both preserving process identity and exception context, and that if it wasn't considered important that effort would not have been made in the first place.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
You could do this better by saying e.g.<br>
<br>
p terminateOn: Error do: [:ex| ex return "skip it"]<br>
<br>
and have terminateOn:do: et al install the exception handler at the<br>
bottom of stack.<br>
</blockquote>
<br></div>
Well, not really. In a realistic situation you'd probably want to handle multiple conditions, timeouts etc. and stitching the stack together for that is likely way beyond your average Squeaker :-)<br>
<br>
[BTW, I don't get why you call this "better"; it would seem that if you call this "better" because it's using in-process termination that your reasoning is cyclical]</blockquote><div><br></div>
<div>Better means it's a form which would work with either approach.</div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="im">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
There are obviously issues if the unwind actions refer to process<br>
local state; but then again perhaps providing said context for the<br>
duration of the unwind is the right thing, along the lines of:<br>
<br>
Process>>terminate<br>
<br>
self isActiveProcess ifFalse:[<br>
Processor activeProcess evaluate: [self unwind] onBehalfOf: self.<br>
].<br>
<br>
This for example would guard errors during unwind as initially<br>
intended while ensuring that process relative state is retrieved<br>
correctly.<br>
<br>
<br>
Yes, but what if I do something like<br>
<br>
[processMemo add: Processor activeProcess.<br>
self doStuff]<br>
ensure: [processMemo remove: Processor activeProcess]<br>
?<br>
</blockquote>
<br></div>
It would work fine, why? Did you not notice that I used #evaluate:onBehalfOf: in the above for precisely this reason?</blockquote><div><br></div><div> OK. But evaluate:onBehalfOf: only fixes one of the two issues. It fixes process identity but it doesn't handle exception context. [BTW, I wrote evaluate:onBehalfOf: only to get process identity right when simulating execution in the debugger; I didn't envisage it being used here, but I agree it would be an improvement].</div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class="im">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
It may have facets but that doesn't mean that running unwinds in other<br>
than the current process isn't badly broken, and isn't relatively easy<br>
to fix ;)<br>
</blockquote>
<br></div>
I don't buy this, and claiming it repeatedly doesn't make it any more real unless you put up or shut up. I think you're vastly underestimating the resulting fallout (does that sound like something I've said before? ;-)<br>
<br>
For example, here is an interesting one (and please use VW for reference here if you'd like; I'd actually be interested in finding a *small* download myself to try some of this stuff): One of the issues in Levente's example of terminating a process halfways through an unwind block is that if the unwind block is ill-behaved you can really be in for trouble. Let's say you have this:<br>
<br>
[<br>
"just to have something to interrupt"<br>
[(Delay forMilliseconds: 100) wait. true] whileTrue.<br>
] ensure:[<br>
"this is the actual killer"<br>
[(Delay forMilliseconds: 100) wait. true] whileTrue.<br>
].<br></blockquote><div><br></div><div>The issue of well-behavedness is orthogonal to in-process or out-of-process termination. We can know whether an unwind block has been started or not and on repeating an attempt to terminate, e.g. after a timeout, one can choose whether to restart an in-progress unwind or skip it and start with the next one, or skip them altogether and abort unwinding. But these choices are not tied to in-process or out-of-process termination except perhaps in implementation details.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">If you wish to support completing a well-behaved ensure block that has been started, it seems tricky to see how this process would actually be terminated, and how you'd find out that it didn't. In Squeak it's simple - the fact that the process sending #terminate is still in this code tells you it's not finished so one can (for example) associate a timeout with #terminate and catch that. Or, one can just interrupt that process and close the debugger. Both are quite reasonable options to deal with the invariant.<br>
</blockquote><div><br></div><div>In VW an unwind block is always executed from a marked sender method so one can find out whether an unwind is running by looking at its sender:</div><div><br></div><div>BlockClosure methods for unwinding</div>
<div><div>valueAsUnwindBlockFrom: aContextOrNil</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"Unwind blocks are evaluated using this wrapper.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>This method is marked as special. When the</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>system searches for unwind blocks, it skips over</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>all contexts between this context and the context</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>passed in as an argument. If the argument is</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>nil, it skips over the sender of this context.</div>
<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>The purpose of this is that errors inside an</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>unwind block should not evaluate that unwind</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>block, and they should not circumvent the</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>running of other unwind blocks lower on the</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>stack."</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span><exception: #unwindInAction></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>| shouldTerminate |</div>
<div><span class="Apple-style-span" style="white-space: pre; "> </span>"long comment elided"</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>shouldTerminate == nil ifTrue: [shouldTerminate := false].</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>self value.</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>shouldTerminate</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ifTrue:</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>[Processor activeProcess terminate].</div></div><div><br></div><div>MarkedMethod methods for testing</div><div><div>isMarkedForUnwindInAction</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"Answer true if method is marked for unwinding in action."</div>
<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>^markType == #unwindInAction</div><div><br></div><div>and exception handling skips over these when looking for handlers.</div><div><br></div>
<div>Context methods for handler search</div><div><div>skipOverUnwindingBlocks</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"If the method is marked for unwindInAction, check the first argument </div>
<div><span class="Apple-style-span" style="white-space: pre; "> </span>of the method. If it is nil, we just skip to the next context in the chain</div><div><span class="Apple-style-span" style="white-space: pre; "> </span>which will be one of #ifCurtailed: or #ensure:. If it is not nil, we skip to</div>
<div><span class="Apple-style-span" style="white-space: pre; "> </span>the context's (represented by the argument to the method) sender."</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>^self method isMarkedForUnwindInAction</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>ifTrue: [(self localAt: 1) == nil</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ifTrue: [self sender sender]</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ifFalse: [(self localAt: 1) sender]]</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>ifFalse: [self]</div><div><br></div><div><br></div><div>and termination by another process (which causes it to execute auto-termination) skips over them also to avoid restarting unwinds already in progress. [Note there's a per-process interruptProtect semaphore used to coordinate termination].</div>
<div><br></div><div><br></div></div><div><br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Under the assumption that you'd like to support the well-behaved case, what do you do when using in-process termination in a non-well-behaved case? How do you find out that a termination request didn't finish, and what do you do in practice to kill it?<br>
</blockquote><div><br></div><div>I would expect to be able to write something like</div><div><br></div><div> [process terminate]</div><div> valueIfLongerThanMilliseconds: 1000</div><div> evaluate: [process terminateWithExtremePrejudice]</div>
<div><br></div><div>i.e. nuke it if it doesn't terminate within a specific time, or</div><div><br></div><div> [[:exit|</div><div> [process terminateAbortingCurrentUnwind]</div><div> valueIfLongerThanMilliseconds: 100</div>
<div> evaluate: exit] valueWithExit.</div><div> process isTerminated] whileFalse</div><div><br></div><div>i.e. abort any unwind if it doesn't execute within a specific time but at least attempt to run all unwinds.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Also, what are the priority implications of the different schemes? One of the more interesting issues here is that in cases like on our servers I could see how one of the lower priority processes times out due to starvation, is then terminated but this may never complete if it's run from the priority it's at instead of the terminator process. Does for example VW bump the priority of the terminated process? How does that interact when the terminated process isn't well-behaved and runs wild?<br>
</blockquote><div><br></div><div>Priority inversion is again an orthogonal issue. It would be great to have priority boosting semaphores. But in any case one could write</div><div><br></div><div> process</div><div> makePriorityAtLeast: Processor activePriority;</div>
<div> terminate</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">I think there are numerous interesting aspects that we have somewhat reasonable answers in Squeak today that I don't even know what the implications would be if you'd go for in-process termination.<br>
</blockquote><div><br></div><div>Do you agree that the right approach is one of specification? What is the correct behaviour and how close to implementing that can one reasonably get? I think from a specification POV one wants both process identity and exception context to be respected when running unwinds during termination and the more straight-forward implementation is to arrange that a process terminates itself when requested to terminate by some other process.</div>
<div><br></div><div>Orthogonal issues as to what happens to unwinds in progress when repeated termination requests are made (from within the process or without) and what priority termination runs at should be specified and then implemented. It's reasonable for the implementation to fall short of an ideal if documented. It's at least regrettable to have the system specified by an implementation.</div>
<div><br></div><div>As far as the implications of in-process termination I don't know of a better way, beyond being able to inspect all current Squeak applications, of voicing a specification and getting people who have written applications affected by this functionality to comment on the specification.</div>
<div><br></div><div>I think I get why the Squeak implementation is a it is (there's a /lot/ of implementation work involved in adding unwinds to the language and it's unreasonable to expect that all of this work will get done at the first attempt). But the fact that the implementation is not ideal shouldn't discourage us from improving it over time.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
Cheers,<br><font color="#888888">
- Andreas<br></font></blockquote><div><br></div><div>best</div><div>Eliot </div></div><br>