[Newcompiler] Trees, sequences, source recovery etc.

[Andrew Tween]

Hi All,
Here is an un-developed idea I had a while ago when I tried to make Shout do
source formatting. It wasn't possible because I only had a sequence of tokens,
no tree.

The idea I had, which I didn't pursue, was to have a lightweight tree, which
referenced the tokens in the sequence. That way I wouldn't replicate the data.

Then I thought about including whitespace tokens in the sequence, making the
token sequence a linked-list (with each token having next/previous pointers),
and also each token having a pointer to its parent tree node.
Since the tree nodes also point to their tokens in the linked-list, it is
possible to navigate up and down the list, hop across to a tree-node, visit its
child nodes, hop up to its parent, and hop back to the list.

(This technique is used in some HTML DOMs , where sometimes you wish to treat
the html as a sequence of <tags>/TEXT, and sometimes as a hierarchy of
<tag></tag>.)

Would something like this help in the storage of comments, whitespace etc?
Would it help with the re-creating of the original source?
Are there any other uses for it?

Cheers,
Andy