Hi Ian,

remember the idea of replacing CompiledMethods in MethodDictionaries
with closures?

More thoughts on that direction.

There are three main representation of executable stuff, AFAIK.
They differ in three main aspects - structure, human useability, machine
usability (efficiency)

Text - freeform, human readable, not directly executable.
ParseTree - structure represents code semantics, not human readable but
can be reasonably represented, requires only a little context to be
runable.
CompiledMethod - a VM datastructure that specifies execution, not human
readable at all and hard to access via messages, but directly executable
by Interpreter.

If we look at Squeak, two of the three form are clearly dominant, and
much more visible to the user - the first and the third.
The user directly edits Texts all the time, and can find CompiledMethods
by inspecting any class.
ParseTrees are mere shadows, temporary constructs built by the Compiler,
and thrown away.
(if you're not sure I'm right about this, in a standard distribution
image
CompiledMethod allInstances size => ~13,000
MethodNode (the root of ParseTrees) allInstances size => ~1 )

The point I'll make is that this is all wrong. ParseTrees should be
dominant, the others mere shadows.
I'll propose some changes, and ask some feasibilty questions.

1. Text is not really a very good way for users to program.
It's true, it is superficially easy (the UI is the same like working
with simple, non executable text).
But right now, it doesn't give much information back. Decorated Text
(eg, format with left-shift held), which can include more information,
is rare in the system right now, and doesn't give much more.

Graphical ways of visualizing program execution might be more useful
even than Decorated Text. But that's not the point - the point is that
both
Decorated Text and Visualization of the program are manufactured from a
ParseTree. One might say, they are simply alternative ways to render
or specify a ParseTree.

So why is so much focus on the source code, and so little on the
ParseTree?

2. On the system side, one might call CompiledMethods a premature
(though great at the time) optimization. The direction of
DynamicPrimitives,
and JIT compilers is to give us more efficient ways to represent code.
In addition, we find that we sometimes want to modify the execution
environment.
We already have a system simulator, and we've seen we might want to
stretch that in other directions - reversible debugging, for example.

What am I proposing? make classes have dictionaries of selector ->
ParseTree (Concretely, MethodNode, or we might choose to wrap it) pairs.

At runtime, the ParseTree can be dynamically decorated by other
representations -
selector -> ExecutingDecorator-> ParseTree
where the decorator will only take over functions it does better
(execution). Note that this would make it easy to experiment with lots
of options,
for example, to setup a fast-compiler/optimizing-compiler pair like in
Self.

3. If we already make execution much more transparent, implementing
extensions like MethodWrappers and such should become much easier.
Refactoring operations become quite natural, as does having a ParseTree
"reattach" - bind itself to a new class, instead of it's old context. So

aClass when: aSelector do: aBlock
    aBlock reattachToClass: self
    self atSelector: aSelector putCode: aBlock.
becomes pretty easy. To me that suggests that blocks also stay mere
ParseTrees until they're used, but then that's only natural...

Possible problems -
The size of the ParseTree representation  - bytecode is pretty small,
but we can keep rarely used trees compressed, and open them on access.
Efficiency of dynamic translation - if we have a method cache, and we
keep that always equiped with directly executable code, we probably will

not have a problem. Self worked using something like this, though I'm
not sure what their intermediate representation was like.

What do you think?

I'd like to try to do a simple version of this (that simply converts
MethodNodes to CompiledMethods). Do you have tips on navigation in the
VM
code for dispatch and such?