Here is my schema, such as it is thus far, for improving the source  
referencing. It's not complete. I need some suggestions for ways to  
tackle a few items.

The problem we face is that a lot needs to be changed in order to use  
anything other than indices into file; so many facilities rely upon  
it. Some serious refactoring would be needed in assorted changelist,  
version listing etc methods. There are complications in ImageSegment  
code too. RemoteString is pretty yucky. Class comments are also mixed  
up in file/index  encoding assumptions. All the source compression,  
tempname-in-method etc code will need altering.

During the writing (and indeed installing later) of the code we need  
all our normal tools to keep working so we can stay sane while doing  
the writing.

The basic idea is to add a proper oop for a source reference object  
so that we can later implement classes that use a database, a web  
search, access multiple files, decompile, guess or whatever.  
Initially I propose simply using an integer with a simple encoding  
scheme (not the rather covoluted one currently in use - take a look  
at StandardSourceFileArray>fileIndexFromSourcePointer: and friends)  
and just reusing the files.

First possibly contentious idea
========================
Do this work in a 3.8.1 image to avoid the change in sources/changes  
files done during the 3.9 cycle and some issues with the introduction  
of Traits. I am *not* suggesting abandoning all the hard work in 3.9  
and anyone implying that I did so will get a late night visitation.  
You Don't Want That.
*After* incorporating the improved source referencing, add the 3.9  
packages but leave out the source condensing step(s). This would  
leave the SqueakV3.sources file untouched and could either leave us  
with a changes file that is simply appended to (and obviously quite  
big) or slightly reformatted (and still pretty big).

Step 1
Add a new source reference ivar to the method properties object that  
all methods now have. Well, except that method properties were added  
somewhere during 3.9 - so they'd have to be added early. Drat. Ideas  
on the minimum disruptive way to add this?

Step 2
Add new method creation methods that do not use the trailerBytes  
stuff. We are not using them yet...

Step 3
Change source access method(s) to check the value of the source  
reference in the methodProperties and use the 'old' access if it is  
nil - which of course it is for now. Also change endPC similarly.

Step 4
change #generate methods to refer to a global flag to say whether  
they use trailerbytes or not. Set the flag *before* that... DAMHIKT!

Step 5
A big do-it to flip the global flag, recompile all methods and  
thereby set the new shiny source pointers.  This is where things can  
get very complicated. We have to decide what to do about the changes  
file and all those historical versions.
a) we could effectively condenseSources so that there is no need to  
worry about handling encrypted pointers to older versions. This would  
of course mean a new .sources file.
b) we could condenseChanges so that any methods in the changelog that  
are also in the old sources file have a correct back pointer, thus  
losing most but not all history.
c) we could try to be very clever and copy the entire history chain  
for each method across to a new changes file, keeping history.
d) we could be insane and try to make code that works out when a  
history pointer is 'old format' and still handle it and then just  
append new format sources to the current changes. I'm not going to  
write that one myself...
Ideas?

Step 6
change the #generate methods to ignore trailerbytes completely, drop  
the global flag and remove it
remove all the redundant method creation methods  etc

Step 7
rework ImageSegment, source code compression, abandonSources, etc  
etc. Ideas?

There is a *lot* needing changing to do this well. I'm horrified how  
poorly factored and written some of this core code is and how it has  
just accreted more and more crap over the last few years. Yuck. Yuck.  
Bad taste in mouth.

tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
Strange OpCodes: EFBI: Emulate Five-volt Battery Intermittently