second call for feedback on Naiad design
Craig Latta
craig at netjam.org
Sat Nov 22 01:48:17 UTC 2008
Hi--
Thanks for the comments! I'm responding to the comments so far in
this single message. I see no reason to restrict Naiad-related
discussion to a single thread; hopefully threads will emerge around
particular specific issues, rather than particular people. :) Please
feel free to break issues out into new threads... for this message,
there's such a grab-bag going that I decided to deal with it all in one
place.
Karl writes:
> When this system works, won't image size be an issue, like an
> ever-growing web browser cache that has no size limit?
I imagine the history memory will have various utilities, like:
- dumping all the compiled method info, because the subject memory
will always have a compiler
- dumping all the method source, because the subject memory will
never have a compiler :)
- storing its less-frequently-accessed editions in one or more
separate history memories, which spend most of their time as
suspended snapshot files, but which can be activated when
necessary. Remote message-sending is a fundamental part of
Spoon; there's no inherent reason why the history memory can't be a
federation of history memories instead.
Of course, one might decide to put editions in another object
database at any point instead (e.g., Magma or Gemstone). I just
want to provide something that provides the bare minimum
functionality "out of the box".
- purging certain editions entirely (rather like when we made new
sources files with the traditional setup)
***
Wolfgang writes:
> For me the main issue is the protocol that is used between the two
> images (subject and history). There is little written about it.
This is true, I haven't finished that documentation yet. One can
look at the implementation of remote message-sending from the last Spoon
release, but I haven't described it in prose yet, and the Naiad design
document is the most prose I've written about how the subject and
history memories communicate at a higher level. Eventually all this
stuff will be in the Spoon book[1].
> Just for thought, what if the history memory would be a web server.
> What would the protocol look like?
Well, there is already a (tiny) webserver in the subject memory,
to provide the initial user interface when first run. One could load
its conveying module into the history memory and do lots of interesting
things with it, yes.
> Can the low-level protocol be hacked to support this?
Yes.
> And one thing I am suspicious is that there is so much knowledge in
> the IDs.
Since they're going to be flying back and forth over sockets,
sometimes in large numbers, they need to be as small as possible; so
I've thought carefully about minimizing them. At this point I'm simply
open to discussion about what anyone would leave out. :) I think have a
good argument for every bit in every ID (likewise for every bit in the
minimal subject memory).
> And limits to the maximum number of editions etc.
So far I've decided that it's not worth any extra bits expressing
variable-length sizes, but again I'm open to discussion about that.
> I'd rather have proper objects that those IDs, with LargeIntegers :-)
(The size argument applies here, too.)
***
Michael writes:
> I think the main reason people aren't commenting is because that's a
> lot of reading!
Sure, I'll just keep asserting that the importance justifies the
time. :)
> Perhaps "versions" is a better name than "editions"? That's the name
> we're more familiar with.
In this case I think the familiarity is a disadvantage; "version"
has multiple strong meanings to people. A "version" is sometimes an
artifact which has multiple interesting states over time, and sometimes
it's an identifier used to refer to such an artifact. I think it's
better to use a less-used term here, and I like the resonance between
"edition" and "edit".
> Do we need to run two instances of Squeak to edit code, one for the
> current version and one for managing the edit history? I assume that's
> what you mean by needing two object memories.
That's right. The typical case is one person using one subject
memory connected to one history memory that is mostly that person's
editions, over a localhost socket connection.
> If so, is it intended for the edit history object memory to be a live
> central repository shared by developers?
That's also an option, yes; it's just not the default.
> Does the system work if it can't contact the edit history object
> memory?
Yes, but the tools would show decompiled method source, and some
history features like regression would be unavailable, (similar to what
happens if you don't have the changes/sources files with the current
setup). But the typical case is that you have the history memory
snapshot on the local machine, so it seems no more likely this would
happen than it would be for one to lose the old changes/sources files,
or indeed the subject memory itself.
> What do your remote references look like?
Each one is an object which holds a special hash for a remote
object, and stream on a socket connected to the remote system. So...
> How stable are they? Do they rely on, e.g. IP address to find a remote
> object memory? If somebody changes IP, are the remote references still
> valid?
...currently, they do not survive suspension or termination of the
object memory in which they live. They are *not* like URLs, as your
comment implies. They are not a description of how to reach a remote
object, they are an active connection to a remote object which behaves
in all ways like the remote object. In general, they are created by
sending messages to other remote objects. The first remote objects in a
session are created specially as part of the connection handshake
between object memories.
If the object memory of the reference is suspended (saved and
quit), the reference is nilled on resumption of the memory.
I implemented this part of the system in 2003; it's been in all
the Spoon releases so far.
> I assume a class now contains a ClassID and a collection of MethodIDs?
No, a class has a "base ID", which is a UUID. The subject memory
as a whole also has a UUID. The history memory knows the UUID of the
subject memory it is tracking, and has "class editions" for each of the
classes that have ever existed in the subject memory. Each class edition
has "method editions" for all of the methods which have ever existed for
that class as defined at a certain point in time.
> Why is ClassID so complex?
It's complex? It's just a base UUID, an author UUID, and a version
number. I think if it were any simpler we'd lose something important.
> Why not just assign each class a new UUID for each new version of that
> class, with authorship and versioning being metadata of that class?
It seems to me that it would be useful to have a single unique
identifier that can refer to the definition of a class at all points in
time, as expressed by all authors. When you want to get more specific as
to author and point in time, you can append additional bits to it.
Also, I explicitly want to keep history information separate from
the artifact objects they describe, so that they may be easily left
behind during production.
> Limiting to 65,536 versions per author is going to create problems in
> 10 years time.
I disagree. Remember, these are editions of a class *definition*
(instance variable format, etc.). If you add a method to a class, you're
not creating a new edition of that class, you're merely creating a new
method edition. From my experience (which encompasses more than ten
years ;), authors tend to create entirely new classes much more often
than they revise class definitions, and they simply use the classes as
they exist a lot more often than that. Frankly, I'd expect 1,024 to
suffice here. Sixteen bits is simply the first sufficient number of
bytes, so it's convenient as well.
> Isn't having the author and version in the [class] IDs going to cause
> conflict problems? What happens if the author is careless and ends up
> with two different versions of a method with the same unique
> identifier?
Another good reason for keeping the history information in a
separate (and headless) object memory is so it can take of itself
without most developers bothering with it. :) The typical developer
uses tools in the subject memory. Those tools only make requests to the
history memory for new editions to be added; they have no say in how the
corresponding IDs are made. In particular, the history memory decides
what the next available version number is for a particular combination
of class base ID, author, and selector.
> Are author UUIDs going to be able to be looked up to get email
> addresses and names somehow?
Each history memory stores author editions; each author edition
associates an author UUID with all that info and more (see the class
tree at [2]). When you receive a module from another author's system,
you get the relevant author editions as well. When you use a new system
for the first time, you can create an author edition for yourself.
> Methods shouldn't have an author. The changes between methods
> versions/editions should have an author.
I disagree. I think it's less work over time to figure out those
changes when necessary.
> I think you're taking the "minimal memory usage" idea too far.
I think it's necessary to make the system as easy to learn and
maintain as I want it to be.
> In my design for distributable packages... packages (cf: classes in
> Naiad)...
I would expect them to correspond to Naiad's modules, not classes.
> ...they need to (deep-)copy it...
Uh-oh... "deep copy" is one of those phrases that immediately
makes me suspect something is wrong (almost as bad as someone saying
"dude" ;).
> I've separated source from bytecodes.
Naiad does that, too.
> I'm not sure it's a good idea to propose an unstable system as the
> next version of Squeak though.
Well, this is two major versions out, not one. I think we can get
plenty of testing in. And I think we're in serious danger of stagnation
as it is. For better or worse, I think this history stuff is the sort of
thing that has to be done with a relatively provocative step. Sometimes
this is good (insert your favorite Alan Kay quote here ;).
thanks again!
-C
[1] http://netjam.org/spoon/book
[2] http://netjam.org/spoon/naiad
More information about the Spoon
mailing list