[Newcompiler] Finding enough bytecodes for closures.

[Klaus D. Witzel]

On Tue, 01 May 2007 18:42:45 +0200, Marcus Denker wrote:
> On 01.05.2007, at 09:45, Klaus D. Witzel wrote:
>> On Mon, 30 Apr 2007 22:47:05 +0200, Bryce wrote:
>>> Klaus D. Witzel writes:
>>>  > Hi Bryce,
>>>  >
>>>  > as part of your investigation, did you observe any chance to
>>> reduce the
>>>  > burden of having an instance of BlockClosure *and* of
>>> ClosureEnvironment,
>>>  > that'd be interesting.
>>>
>>> First the new compiler only creates a BlockClosure and an
>>> ClosureEnvironment if the block captures variables from it's
>>> surrounding scope.
>>>
>>>    self ifAbsent: [Set new].
>>>
>>> Builds the BlockClosure at compile time. In this case, especially if
>>> the block closure isn't called the closure code will be much faster.
>>
>> Sure. But who wants BlockClosures without having "free" variables,
>> this would be a contradiction and not worth the effort building a
>> new compiler for them :|
>>
>
> But there are many cases of BlockClosures that reference no external
> variables... e.g. along the example given by bryce:
>
> 	someDict at: #test ifAbsentPut: ['hello']
>
> here the block has no variables at all, thus is does not need to have
> an Environment.

Sure (now repeated, how many times? :) There are cases where no Env's are  
needed. But you cannot design+build the new compiler for just the cases  
where no Env is needed?

> For blocks with no free vars, the newCompiler generates the
> BlockClosure at compiletime, see
> ASTTranslatorForValue>>#acceptBlockNode:

This method is my best friend, especially Preferences  
compileBlocksAsClosures ifFalse:ifTrue: :)

> 	aBlockNode freeVars isEmpty ifTrue: [
> 		"Create block at compile time"
> 		methodBuilder pushBlock: blockMethod.
> 	] ifFalse: [
> 		"Create block at run time"
> 		| outerScope envVar |
> 		outerScope := aBlockNode scope outerScope.
> 		envVar := outerScope hasEscapingEnv
> 			ifTrue: [outerScope thisEnvVar]
> 			ifFalse: [outerScope receiverVar].
> 		methodBuilder pushBlockMethod: blockMethod.
> 		envVar emitLocalValue: methodBuilder.
> 		methodBuilder send: #createBlock:.
> 	].
>
> The old compiler generates in all cases code where the block's
> bytecode is inlined in the method and uses
> #blockCopy: to generate a MethodContext at *runtime*.

The above code is really, really a nice solution: everything's in one  
place, comprehensible and easy changeable :)

> Now the story when you actually send #value...
>
> BlockContext: The vm has already a Context, so it fills it up with
> all the runtime data and start executing.
> BlocKClosure: The vm executes a *method* that is referenced from the
> Closure. This means, it needs
> 			  to set up a context. Here there comes in the very clever
> optimization that the VM can
>                            re-use context, thus, if we take care (not
> accessing thisContext), we do not need to create
>                           a context but reuse it.

Exactly (*reuse*, see also below).

> The idea now is that "reusing" a context should not be slower then
> actually using BlockContext, with the
> additional feature that we have the optimzation described above.
> 	

Sure :)

BTW: I've modfied ASTTranslatorForValue>>#acceptBlockNode: to do exactly  
that.

>
>>
>>> Yes it is possible to eliminate many ClosureEnvironments, they're
>>> only
>>> needed when variables are actually captured from that scope.
>>
>> Sure "only if needed" but otherwise nobody would need a new
>> compiler which handles full BlockClosures.
>>
>
> But they are not needed in *all* cases. ['I am a block'] does not
> need an environment for storing variables.

*absolutely* *no* *problem* *and* *never* *seen* *as* *a* *problem* *here*  
(apologies for the emphasis) but that cannot be taken as the general case  
(and just the latter is my point, nothing more). Instead, efficient code  
needs to be generated for precisely the opposite case(s). Is it possible  
we agree on this one?

>>> Removing
>>> the closures is just creating extra kinds of block similar to the
>>> different kinds of context that Eliot added in VisualWorks
>>> 5i. Creating extra context types is probably not needed to equal
>>> or better
>>> the current code in most cases.
>>
>> Yes (extra context types probably not needed to equal or better),
>> me too thinks that's not really needed.
>
> It would be nice to make it simple and uniform.... the current
> BlockContext scheme preceeds the contex-recycling
> scheme...

There's one particular reason why #pushActiveContextBytecode has to cancel  
(as noted by Bryce in the other thread) the recycling possibility of  
contexts: any piece of code could have stored the context (MethodContext  
and BlockContext alike) behind the VM's back, say for example into a slot  
of a longer living object (like in an iVar). From then on recycling is  
impossible.

Can we agree that this situation (cause+effect) cannot change, regardless  
of using MethodContext, BlockContext, BlockClosures or any replacement for  
the latter?

> and having Blocks and Methods be one thing would be nice.

It already does so here, and it reuses the context object when sending  
#value to closures, the same way as it reuses "old" BlockContexts. But it  
does not change the situation of context-recycling :(

Cheers
Klaus

> 	Marcus