DeltaStreams status (Re: [squeak-dev] Let's push it)

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Tue Jun 30 14:37:52 UTC 2009


2009/6/30 Göran Krampe <goran at krampe.se>:
> Hi!
>
> Edgar J. De Cleene wrote:
>>
>> Eras ago I volunteer to test both DeltaStreams and MC2 , but nobody wish
>> risk and we take the Blue Pill (remember?)
>
> Ah! DeltaStreams! Just waited for it to get mentioned. :)
>
> I actually sat last night trying to get it slightly into shape for Igor to
> join in. Quick recap and status:
>
> DeltaStreams is an idea (and code) that I came up with a while back when
> trying to figure out "new ways" to live in a forked world. The idea
> consisted of two parts - Deltas and Streams of them. :)
>
> A Delta is simply a "ChangeSet DeLuxe". Main differences off the top of my
> head:
>
> - Is in fact an *ordered* sequence of changes. (A ChangeSet is a *Set* - no
> ordering)
> - Each change in a Delta is much more fine granular and has *all*
> information captured about the change.
> - Each change can produce its "anti" change.
> - A Delta is a *standalone* object. It does not refer to anything in the
> image, each change contains all its information in its own object graph.
>
> This standalone property means that you first *load* a Delta into the image
> and then *apply* it if you want to. Thus you can load one from disk and look
> at it *without* applying it! You could harvest the Internet for Deltas, suck
> hundreds of them into an image, and then inspect them.
>
> The anti bit above also means that a Delta can be reverted, it just applies
> all its anti changes in reverse order! And yes, this shit actually works. :)
>

Goran, this is great work I'd like to support.
But I can only believe that it works in most cases...
More exactly, I suspect it works for code, but not on lively instances:
If I remove an inst var, all instances of the class and subclasses
will loose some information.
Do you store that information in the delta so that reversion restore the state?
Otherwise, I know of some anti-refactoring that will kill my image nicely.

If I remove a class, how will orphaned instances be restored?

Do DeltaStream address something like refactoring ScaledDecimal from
something like:
Number subclass: #ScaledDecimal instanceVariables: 'fraction precision'.
to:
Fraction subclass: #ScaledDecimal instanceVariables: 'precision'.

This shit is real (MIT refactoring applied in Pharo) and requires some
other shit like:
Create class ScaledDecimalTemp
Mutate allInstances of ScaledDecimal to ScaledDecimalTemp
remove ScaledDecimal
rename ScaledDecimalTemp to ScaledDecimal

I'm very interested how would you handle such mutation and
anti-mutation in DeltaStream?
It's doable with user crafted pre/post scripts without system
validation... Is that the solution?

Nicolas

> The DeltaStreams code uses SystemEditor from Colin Putney - the new
> mechanism to *atomically* apply changes to the image. In fact, our work on
> DS contributed several fixes to SE because DS was first to use it seriously.
> I thin MC1.6 maybe uses SE too?
>
> Deltas are "recording" just like ChangeSets are, but since they record all
> changes in order - they do double duty as a changelog too.
>
> Matthew Fulmer started out implementing a clever hack so that Deltas could
> be saved as .cs files and actually "work" as a ChangeSet if filed in as one.
> Now that I am diving into the code again I have decoupled that file format
> (but still there as an extra package) and instead will implement a format
> using Tirade. I have blogged about Tirade in 3 articles so you can read
> about it there. It is a readable format that looks like Smalltalk code, but
> it is very fast to parse and secure.
>
>
> Status:
>
> The domain code is perhaps 90% covered. There are some small things to take
> care of and since I am now hooking in Tirade I am refactoring some things a
> bit. A lot of tests are green - but not all. I really want to move to a
> fully green state. Experts are more than welcome to help out!
>
> An example of a "problem": If I have a method change that moves a method to
> a new method category - and then I *revert* that change, should the new
> method category (if now empty) be removed? tricksy.
>
> The Tirade hookup is ongoing but not hard work, it should be doable in a
> couple of nights.
>
> Then I wish to implement a DualDeltaSorter :) - simply a "clone" of dual
> changesorter that works with Deltas. If anyone would like to start that
> work, or implement some other cool thing like:
>
> - ChangeSet -> Delta converter. Filing in a ChangeSet should simply create a
> Delta instead of a ChangeSet.
> - MC diff -> Delta converter. Given two MCs we can produce a Delta.
>
> Just holler!
>
> Usecase:
>
> Let's say the Pharo people fixes a bug in Pharo. By the look of it the bug
> should exist in Squeak.org too. Let's say we produce a Delta either from the
> Pharo changeset or from the MC snapshot that contained the fix.
>
> The Delta is filed out from Pharo in Tirade format. It is easily readable in
> a text editor just like a ChangeSet is. You fire up Squeak.org and open up
> the DualDeltaSorter and *load* the Delta into the image. At this point *no
> code* has been touched in the image, we just deserialized the Tirade file
> and created the DSDelta object.
>
> We can now look at the actual changes it contains, perhaps 3 new methods, 4
> modified methods, some ivars added - all over the image.
>
> A Delta is like a "semantic patch". Since it knows more about itself it can
> reason about its own applicability. For example, it separates "modified
> method" and "added method". And it contains both the "before" and "after"
> state of each change. For example a modified method contains both code
> before and the new code!
>
> We thus ask the Delta if it can be "cleanly applied". Cleanly means that the
> DSDelta checks if the state of the image concerning the changes is the same
> as it was in the Pharo image. For example, regarding the modified methods -
> do they look like they did in Pharo *before* they were modified? If yes,
> then we are good to go. Can we add the ivars? Are all expected classes in
> the image? Did any of the new methods perhaps already exist?
>
> Let's say it claims to apply cleanly (there are levels of "cleanliness"
> here, for example a method may already be in the "new state", which is
> probably fine but not perfectly clean).
>
> We apply it (done by SystemEditor atomically). Then we run tests or
> whatever, if it seems to have broken something we could decide to just back
> it out with "revert". We could also set the Delta to record and then try to
> fix it.
>
> Conclusion:
>
> DeltaStreams are not here yet - but pretty darn close to be able to replace
> ChangeSets at least! It just needs a push. This means I agree with Andreas
> to use MC currently, but I also would like us to dig in our heels and get DS
> ready. It should bring us:
>
> - Speed.
> - Lots of flexibility.
>
> regards, Göran
>
>
>



More information about the Squeak-dev mailing list