DeltaStreams status (Re: [squeak-dev] Let's push it)
Göran Krampe
goran at krampe.se
Tue Jun 30 13:58:39 UTC 2009
Hi!
Edgar J. De Cleene wrote:
> Eras ago I volunteer to test both DeltaStreams and MC2 , but nobody wish
> risk and we take the Blue Pill (remember?)
Ah! DeltaStreams! Just waited for it to get mentioned. :)
I actually sat last night trying to get it slightly into shape for Igor
to join in. Quick recap and status:
DeltaStreams is an idea (and code) that I came up with a while back when
trying to figure out "new ways" to live in a forked world. The idea
consisted of two parts - Deltas and Streams of them. :)
A Delta is simply a "ChangeSet DeLuxe". Main differences off the top of
my head:
- Is in fact an *ordered* sequence of changes. (A ChangeSet is a *Set* -
no ordering)
- Each change in a Delta is much more fine granular and has *all*
information captured about the change.
- Each change can produce its "anti" change.
- A Delta is a *standalone* object. It does not refer to anything in the
image, each change contains all its information in its own object graph.
This standalone property means that you first *load* a Delta into the
image and then *apply* it if you want to. Thus you can load one from
disk and look at it *without* applying it! You could harvest the
Internet for Deltas, suck hundreds of them into an image, and then
inspect them.
The anti bit above also means that a Delta can be reverted, it just
applies all its anti changes in reverse order! And yes, this shit
actually works. :)
The DeltaStreams code uses SystemEditor from Colin Putney - the new
mechanism to *atomically* apply changes to the image. In fact, our work
on DS contributed several fixes to SE because DS was first to use it
seriously. I thin MC1.6 maybe uses SE too?
Deltas are "recording" just like ChangeSets are, but since they record
all changes in order - they do double duty as a changelog too.
Matthew Fulmer started out implementing a clever hack so that Deltas
could be saved as .cs files and actually "work" as a ChangeSet if filed
in as one. Now that I am diving into the code again I have decoupled
that file format (but still there as an extra package) and instead will
implement a format using Tirade. I have blogged about Tirade in 3
articles so you can read about it there. It is a readable format that
looks like Smalltalk code, but it is very fast to parse and secure.
Status:
The domain code is perhaps 90% covered. There are some small things to
take care of and since I am now hooking in Tirade I am refactoring some
things a bit. A lot of tests are green - but not all. I really want to
move to a fully green state. Experts are more than welcome to help out!
An example of a "problem": If I have a method change that moves a method
to a new method category - and then I *revert* that change, should the
new method category (if now empty) be removed? tricksy.
The Tirade hookup is ongoing but not hard work, it should be doable in a
couple of nights.
Then I wish to implement a DualDeltaSorter :) - simply a "clone" of dual
changesorter that works with Deltas. If anyone would like to start that
work, or implement some other cool thing like:
- ChangeSet -> Delta converter. Filing in a ChangeSet should simply
create a Delta instead of a ChangeSet.
- MC diff -> Delta converter. Given two MCs we can produce a Delta.
Just holler!
Usecase:
Let's say the Pharo people fixes a bug in Pharo. By the look of it the
bug should exist in Squeak.org too. Let's say we produce a Delta either
from the Pharo changeset or from the MC snapshot that contained the fix.
The Delta is filed out from Pharo in Tirade format. It is easily
readable in a text editor just like a ChangeSet is. You fire up
Squeak.org and open up the DualDeltaSorter and *load* the Delta into the
image. At this point *no code* has been touched in the image, we just
deserialized the Tirade file and created the DSDelta object.
We can now look at the actual changes it contains, perhaps 3 new
methods, 4 modified methods, some ivars added - all over the image.
A Delta is like a "semantic patch". Since it knows more about itself it
can reason about its own applicability. For example, it separates
"modified method" and "added method". And it contains both the "before"
and "after" state of each change. For example a modified method contains
both code before and the new code!
We thus ask the Delta if it can be "cleanly applied". Cleanly means that
the DSDelta checks if the state of the image concerning the changes is
the same as it was in the Pharo image. For example, regarding the
modified methods - do they look like they did in Pharo *before* they
were modified? If yes, then we are good to go. Can we add the ivars? Are
all expected classes in the image? Did any of the new methods perhaps
already exist?
Let's say it claims to apply cleanly (there are levels of "cleanliness"
here, for example a method may already be in the "new state", which is
probably fine but not perfectly clean).
We apply it (done by SystemEditor atomically). Then we run tests or
whatever, if it seems to have broken something we could decide to just
back it out with "revert". We could also set the Delta to record and
then try to fix it.
Conclusion:
DeltaStreams are not here yet - but pretty darn close to be able to
replace ChangeSets at least! It just needs a push. This means I agree
with Andreas to use MC currently, but I also would like us to dig in our
heels and get DS ready. It should bring us:
- Speed.
- Lots of flexibility.
regards, Göran
More information about the Squeak-dev
mailing list
|