[Enh][VM] primitiveApplyToFromTo for the heart of the enumeration of collections?

Klaus D. Witzel klaus.witzel at cobss.com
Fri Sep 15 23:12:20 UTC 2006


Hi Bryce,

reading your comments below I'm under the impression that you have not  
understood the novel concept (which is, BTW old) but, of course, I may be  
mistaken.

And I appreciate your comments and thoughts very much :) In the below I  
offer discussing some issues off list but, I'd happily discuss all issues  
here.

On Fri, 15 Sep 2006 23:44:47 +0200, you wrote:
>
> At the moment no primitives are re-entered.

primitiveApplyToFromTo is always called from the *beginning* with the same  
context. No pc is maintained. No sp is maintained. Of cause both  
quantities are defined for the context, but not touched in any way.

> We don't have primitive
> contexts.

This is not true, all primitives have their arguments in the context and  
also leave their result there, since the Blue Book.

> Changing that is a large design change even if the code
> change is small.

There was no design change, either executeNewMethod is called which in  
turn calls activateNewMethod then newActiveContext, or between  
activateNewMethod and newActiveContext is primitiveApplyToFromTo called.  
The flow of control is 100% pure[tm] Blue Book and untouched.

> Normal primitives can use shadowing because there is
> no way to capture their execution half way through.

No, there is no half way through. primitiveApplyToFromTo is always full  
way through. It is not so that, say 50% of its code is covered by the  
first invocation and the rest by the next invocation. The coverage is  
always 100%.

> In this case that
> is not true. Shadowing does not fully hide the primitive as it may
> need to be re-entered later.

Absolutely not. primitiveApplyToFromTo can send, say, the first 10 values  
to the block and then may fail. Thereafter its shadow can, if it wants,  
send say, another 10 values to the block.

Your comment shows me that the code of primitiveApplyToFromTo is not easy  
to understand. This must be my fault, we can discuss this off list to any  
level you want.

> This primitive will slow down sends and returns on some architectures
> but probably not all.

On which, Bryce?

> Out of order execution is great at hiding cost
> behind other delays.

primitiveApplyToFromTo concatenates (the primitive implementation of)  
Stream>>#next and BlockContext>>#value:, what is out of order with this?

> I wouldn't be surprised if there is no cost on
> either a PPC or an x86 but there will be on some chips.

Which chips, Bryce?

> Any
> optimisation of the common case send code will increase the
> performance loss caused by this primitive. (1)

Until now, you have not shown any performance loss caused by this  
primitive. So what's this about?

> It could crash if you save an image on a VM with the primitive then
> load it on a VM with the image.

It cannot crash. It can start on VM-a *with* primitiveApplyToFromTo  
support, then, before its block ends, the image can be written to a  
snapshot, then resumed on VM-b *without* primitiveApplyToFromTo support,  
and vice versa. Let's walk through this off list.

> This will only happen if one of the
> primitives is in an active context. Looking at
> internalActivateNewMethod the PC will be set to it's initial value but
> that will cause the loop to begin again which could be problematic too.

Every loop begins again because of the long jump backwards. But in  
primitiveApplyToFromTo there is no jump backwards, no loop. And there is  
nothing which causes the loop to begin again. As I wrote earlier, it is
  (*marker*) [index < limit] whileTrue: ...
and there exists no jump back to the (*marker*) position. I repeat: there  
is no jump.

> What happens if you step back into the primitiveApplyToFromTo method
> from a debugger?

The same what happens when the block returns (I mean: indistiguishable,  
invariant).

> So execution entered via the interpreter and used
> the primitive then the debugger (or any tool that simulates bytecode
> execution) re-enters the primitive method.

This is supported. All the debugger must know is that it cannot step  
through the primitive (easy). But it can step through the shadow and then,  
when in the shadow (or in the block) one does "proceed", it is again the  
same as when the block returns (indistiguishable, invariant).

> There is maintenance risk if the shadow implementation and the real
> implementation get out of sync because the bugs may only occur
> when switching from running with the primitive to running without the
> primitive.

You mean like with the shadow of any other primitive? Sure, this always  
was so and will always be so. Two pieces of software which implement the  
same function, but one transcedenting the other, have always this problem.

> This will definitely happen if you move an image to an
> older VM

No, see the case with VM-a and VM-b above.

> but could also happen if you improve the primitive so it can
> be re-entered as bytecode.

Huh? the primitive is not bytecode and it is not planned to make it such.

> Also a primitive failing does not necessarily mean that it can be
> replaced by the back-up code.

No, primitives must be replaceable by their shadow, or else every other  
existing shadowed primitive could crash the VM.

> In many cases the method code after a
> primitive handles a different set of conditions.

Why do you want me to handle conditions in primitiveApplyToFromTo which  
are not handled in the shadow. That would be a bit too much, if not crazy.

> Have a look at
> Object>>size.

Have a look at all the primitives that expect Integer indices but are  
passed Floats. This is business as usual.

Back to our case here, if someone passes a Float to primitiveApplyToFromTo  
then the shadow will attempt to index the receiver with a Float, like in  
(#(1) at: 0.5), so what?

> In general, no execution engine can assume that it can
> ignore a primitive.

Not so fast and not so general: primitiveApplyToFromTo can be ignored by  
every VM which was compiled *with* it and by every VM which was compiled  
*without* it.

Thank you very, very much Bryce :-)

/Klaus

> Bryce
>

I understand the following as a general comment on the current  
implementation of #commonReturn.

> 1) Have a look at commonReturn. The simple case when a method or
> block is returning directly to it's caller can be simplified. The
> general case needs to handle any unwind blocks that might be walked
> over while exiting which the common case will not do. Also the common
> case could be coded without the loops which risk branch mispredict
> when exiting.
>
>





More information about the Squeak-dev mailing list