Hi all--

Now that I've been living with it for a while, here are a few details on the shrinking technique I mentioned a couple of months ago (where methods that haven't been run recently get reclaimed by the garbage collector). Also, at the end, are some notes on what I'm doing now and my next steps.

When I did the earlier imprinting work, I added two bytes to the compiled method trailer format. One bit in those bytes indicates whether the virtual machine has run the method (the other fifteen are for recording the method's linear version). I changed the virtual machine to set that bit every time it runs a method. By clearing that bit on all methods, then examining them later after running the system for some duration, one can tell all the methods which were run over that duration. A method without that bit set is "inert".

I extended the garbage collector with an alternative mark phase that doesn't mark or trace inert methods (with the exception of methods associated with currently-active contexts). References to inert methods are replaced with nil. This leaves some method dictionaries with nils where methods used to be.

My typical development mode so far has been to use a "full" object memory equipped with remote-messaging tools to control a target object memory over a network. This lets me make changes to the target with impunity. In particular, I don't have to worry about the target getting wedged because I've broken the user interface, because I'm using another system's user interface. I've got a remote system browser, debugger, process browser, and workspace. The remote system browser uses the master system's compiler, and transfer methods directly into the target, so the target need not have a compiler (or ClassBuilder). No class names are ever exchanged between master and target, and source code is completely optional.

I changed the target system's Object>>doesNotUnderstand: so that, before raising an exception, it first attempts to install the missing method from the connected master system. If installation is successful, it resends the method and carries on. I changed method lookup in the virtual machine so that when a nil is encountered where an inert method used to be, it is handled as a message-not-understood. So, the master system effectively acts as a virtual memory for the target, providing missing methods inline as they are encountered. In the past (before the garbage collector changes), a new target system was created as a copy of the master system. I would then connect master and target, and use the remote tools in the master to shrink the target manually. I've done two passes that way; the first took three months of work in 2003, the second took two weeks of work in 2005. The reason I've made multiple passes is that I kept realizing significant system features I'd forgotten to include before I started shrinking (e.g., remote debugging), and it'd be much more work to retrofit it into the target than to just shrink a new master copy. Since there will probably always be new fundamental things to add, I decided I ought to just make the shrinking process more automatic.

Now I've got a class in the master that can automatically gut the target in 30 minutes. It invokes code in the target that can throw out entire "inert classes" (classes which have no references and no non-inert methods). I got rid of the entirety of Morphic this way, for example, without having to understand anything about how Morphic works.

***

In January I wrote a module-aware webserver for the target, to which the web-based Spoon installer will redirect the user after downloading and starting the system. The user will then be able to discover and select modules to load and unload, make snapshots, quit, etc. Currently I'm working on a Naiad module which installs the ClassBuilder (for manipulating existing classes in the target). (Naiad is Spoon's module system.)

After that I plan to:

- make a module which installs every last bit of 3.8 final, just to show that one can recreate familiar old systems

- make modules which install the VM construction tools

- make a VM from a Spoon system with those modules in it

- throw away the VMs and object memories that I started with!

- make modules for other things I want to use (Quoth, Chronos, Weather on Display, Tweak, etc. etc.)

Ideally, I'd like the new modules to contain well-factored and highly-readable expressions of the ideas of the old subsystems, rather than just blind repackagings, but we'll see... it's tempting to just imprint things to save time. :) (For more about imprinting, see [1].)

thanks,

-C

[1]

http://lists.squeakfoundation.org/pipermail/spoon/2004-October/000061.html