DeltaStreams status (Re: [squeak-dev] Let's push it)

Göran Krampe goran at krampe.se
Tue Jun 30 13:58:39 UTC 2009


Hi!

Edgar J. De Cleene wrote:
> Eras ago I volunteer to test both DeltaStreams and MC2 , but nobody wish
> risk and we take the Blue Pill (remember?)

Ah! DeltaStreams! Just waited for it to get mentioned. :)

I actually sat last night trying to get it slightly into shape for Igor 
to join in. Quick recap and status:

DeltaStreams is an idea (and code) that I came up with a while back when 
trying to figure out "new ways" to live in a forked world. The idea 
consisted of two parts - Deltas and Streams of them. :)

A Delta is simply a "ChangeSet DeLuxe". Main differences off the top of 
my head:

- Is in fact an *ordered* sequence of changes. (A ChangeSet is a *Set* - 
no ordering)
- Each change in a Delta is much more fine granular and has *all* 
information captured about the change.
- Each change can produce its "anti" change.
- A Delta is a *standalone* object. It does not refer to anything in the 
image, each change contains all its information in its own object graph.

This standalone property means that you first *load* a Delta into the 
image and then *apply* it if you want to. Thus you can load one from 
disk and look at it *without* applying it! You could harvest the 
Internet for Deltas, suck hundreds of them into an image, and then 
inspect them.

The anti bit above also means that a Delta can be reverted, it just 
applies all its anti changes in reverse order! And yes, this shit 
actually works. :)

The DeltaStreams code uses SystemEditor from Colin Putney - the new 
mechanism to *atomically* apply changes to the image. In fact, our work 
on DS contributed several fixes to SE because DS was first to use it 
seriously. I thin MC1.6 maybe uses SE too?

Deltas are "recording" just like ChangeSets are, but since they record 
all changes in order - they do double duty as a changelog too.

Matthew Fulmer started out implementing a clever hack so that Deltas 
could be saved as .cs files and actually "work" as a ChangeSet if filed 
in as one. Now that I am diving into the code again I have decoupled 
that file format (but still there as an extra package) and instead will 
implement a format using Tirade. I have blogged about Tirade in 3 
articles so you can read about it there. It is a readable format that 
looks like Smalltalk code, but it is very fast to parse and secure.


Status:

The domain code is perhaps 90% covered. There are some small things to 
take care of and since I am now hooking in Tirade I am refactoring some 
things a bit. A lot of tests are green - but not all. I really want to 
move to a fully green state. Experts are more than welcome to help out!

An example of a "problem": If I have a method change that moves a method 
to a new method category - and then I *revert* that change, should the 
new method category (if now empty) be removed? tricksy.

The Tirade hookup is ongoing but not hard work, it should be doable in a 
couple of nights.

Then I wish to implement a DualDeltaSorter :) - simply a "clone" of dual 
changesorter that works with Deltas. If anyone would like to start that 
work, or implement some other cool thing like:

- ChangeSet -> Delta converter. Filing in a ChangeSet should simply 
create a Delta instead of a ChangeSet.
- MC diff -> Delta converter. Given two MCs we can produce a Delta.

Just holler!

Usecase:

Let's say the Pharo people fixes a bug in Pharo. By the look of it the 
bug should exist in Squeak.org too. Let's say we produce a Delta either 
from the Pharo changeset or from the MC snapshot that contained the fix.

The Delta is filed out from Pharo in Tirade format. It is easily 
readable in a text editor just like a ChangeSet is. You fire up 
Squeak.org and open up the DualDeltaSorter and *load* the Delta into the 
image. At this point *no code* has been touched in the image, we just 
deserialized the Tirade file and created the DSDelta object.

We can now look at the actual changes it contains, perhaps 3 new 
methods, 4 modified methods, some ivars added - all over the image.

A Delta is like a "semantic patch". Since it knows more about itself it 
can reason about its own applicability. For example, it separates 
"modified method" and "added method". And it contains both the "before" 
and "after" state of each change. For example a modified method contains 
both code before and the new code!

We thus ask the Delta if it can be "cleanly applied". Cleanly means that 
the DSDelta checks if the state of the image concerning the changes is 
the same as it was in the Pharo image. For example, regarding the 
modified methods - do they look like they did in Pharo *before* they 
were modified? If yes, then we are good to go. Can we add the ivars? Are 
all expected classes in the image? Did any of the new methods perhaps 
already exist?

Let's say it claims to apply cleanly (there are levels of "cleanliness" 
here, for example a method may already be in the "new state", which is 
probably fine but not perfectly clean).

We apply it (done by SystemEditor atomically). Then we run tests or 
whatever, if it seems to have broken something we could decide to just 
back it out with "revert". We could also set the Delta to record and 
then try to fix it.

Conclusion:

DeltaStreams are not here yet - but pretty darn close to be able to 
replace ChangeSets at least! It just needs a push. This means I agree 
with Andreas to use MC currently, but I also would like us to dig in our 
heels and get DS ready. It should bring us:

- Speed.
- Lots of flexibility.

regards, Göran




More information about the Squeak-dev mailing list