On Saturday, February 8, 2003, at 02:02 PM, Mark S. Miller wrote:
Rob, I admit I'm a bit confused by your message. Earlier, you wrote:
Mark, I'll admit that I am confused, myself. I am confused about what the right way forward is, when contemplating the changes we need to make to reach level 4.
I still support getting to language-based security, rather than just framework-based, and I realize that this won't be possible without breaking things.
Joshua mentioned the failed modules effort. Part of the reason it failed was because it changed some basic assumptions in the squeak environment and this made it unusable. It left the community behind. The other concern is providing enough functionality so that the people here, for instance, may be able to start using it and making it better. I would love it if that were to happen!
So I am not retreating, but trying to find the balance between having our environment and extending it with inabilities. :) I would like to rename your 'A Coherent Retreat' proposal to 'A Functioning Interim Vat', since I am making good headway on CapTP. It is the area where I can effect some benefit.
I personally am bound into an implementation perspective and I don't have knowledge or experience in the areas being discussed, around scoping and the compiler. Hopefully my comments aren't a distraction to the discussion. I also have some limited time in which to finish what I can, and hopefully incorporate it into squeak. My limitation is that I am working full-time, while doing SqueakE work on the side. In April, sailing season begins again and I expect that I will be on the water every weekend until October, so I won't have but a couple of hours a week to work on this through the summer, and that isn't the kind of time that would allow me to do deep squeaking.
This does not mean that I am opposed, in the long run, to the burn-the-diskpacks approach, only that it should be tempered with the realities of community building, therefore there are several milestones I am thinking in terms of.
First milestone is to see how integrated we can make the eventual sending mechanism into core squeak. This is my goal for April. This requires Andreas' ScriptProcessor, so that we can integrate eventual sending with the average user of Semaphores and sequential scripts. Andreas, if you have a more current version of your core scripting framework, I'd love to see it.
Second milestone is the virtualization layers to support debuggability.
Third is the compiler and language that would provide level 4 security.
There is no reason that they shouldn't be concurrent efforts, but we should point to that feature (system openness) as so important that it deserves its own milestone sooner rather than later.
On Saturday, February 8, 2003, at 06:08 PM, Joshua 'Schwa' Gargus wrote:
I'm very interested in Squeak-E, and would like to incrementally modify my code to fix whatever breaks as APIs are tamed. But I _need_ the programming environment that I am accustomed to; if Squeak-E can't support the debugger and other development tools, then I fear that the Squeak community will not follow.
This is the critical point when a burn-the-diskpacks approach is usable. When it supports the investigation tools to allow us to manipulate it, it will be more broadly usable. Anything we can do to continue working in an environment that currently supports these things, we should take advantage of. This means an approach like Andreas has suggested seems required. This approach will allow us to fundamentally change the language, while still working in our development environment.
On Saturday, February 8, 2003, at 02:02 PM, Mark S. Miller wrote:
I think that, by doing this, we can eventually build back up to something close to the level of functionality of current Squeak. But the result will not be compatible. You cannot add inabilities to a system the way you add a new package. New inabilities means that some old code must break.
When we do this, we should make every attempt to not "lock-down" the security so we can debug it with normal tools, until we can support a debugger and inspector of the right's amplification variety. This tells me that one of the first things we do after getting CapTP and Eventual Sending done, is to build your virtualization layers. :)
We really have two orthogonal directions to take. First is the execution environment and support for tools. The other is the compiler and mutability and plugging leaks in the global scope. If you agree that this seems to be a natural fault line in the work ahead, then perhaps we should form two sub-efforts. The first can look at how to build an inspector and debugger, while the second look at the new Object hierarchy rooted at EObject and it's compiler.
This process -- removing everything that doesn't follow capability discipline, followed by adding back in new APIs that provide similar functionality in a capability-compatible way -- is a kind of refactoring, though a much more violent case then we normally encounter. A great example is Lex's ObjectInspector in the Island's work. It moves the reflective messages from the objects themselves, where any client of the object could invoke them, and puts them instead on ObjectInspector, which can be given only to those that should indeed be authorized to break the encapsulation of these objects. (The Brand system explained earlier enables the issue of "which objects" to be dealt with in a finer-grained fashion, but that's not relevant to the points I make here.)
That's easy enough. We have to change the MOP for our tools. :)
I have been rereading your post on the KeyKOS brand and your comments that this could be placed in a register. In other posts it almost seems that you are separating the lexical scoping from the code. I am trying futilely to separate the different dimensions of an execution environment. You have introduced the concept of ownership into the object graph and the idea that the scope is fixed at the time of creation of an object, rather than retrievable by walking the stack. I can't begin to say that I understand this.
When the ideas are finally understood to the point where there is an implementation with an inspector and debugger, then we would have a usable environment.
If you think you can get to capability security in an upwards compatible way, what are you going to do about #instVarAt:? Or the APIs for opening and writing any of the user's files? Do you really want any Squeaklet to be able to do that?
Hopefully the MOP replacement for #instVarAt:, will allow us the same functionality.
A Coherent Retreat
There is another kind of capability security you can achieve with full upwards compatibility. It is indeed quite useful, it is the one you've already made wonderful progress on, it is where E started and stayed for years, it is the way E's largest customer is using it, and it is also the kind of security supported by Waterken Software http://www.waterken.com . If indeed the Squeak community is not willing to take the steps needed to design and build a Squeak-like capability secure *language*, then this may indeed be a good place to retreat to. In any case, it's something you need to build anyway, it's something you are building anyway, and it may be a good place to start.
The fact that this is 1) valuable and 2) something we need anyway, tells me that we ought to make it a first step on the way to the broader changes. That we may be able to do this and maintain mostly backward compatibility is encouraging to me.
What you give up is language-based security. Nothing above leverages the almost-capability nature of Smalltalk. You wouldn't be achieving anything that couldn't be achieved equally well by C++ programmers using a CapTP library written in C++. You'd have no ability to locally run code you don't trust, and therefore no ability to make Squeaklets happen.
This is still a goal, but we need to preserve our environment to get there.
So if this is indeed done as a retreat, it would sadden me. I think that a Squeak-E *language* would be a wonderful thing.
I think it would be too, but it is a long way off. As we strive to get there, let's make sure we keep it a running system.
In the kind of massive refactoring we'd need to get language-based security, not only can't we be upwards compatible, we can't necessarily retain all the features we'd like. Our likes evolved without constraint of security. Some things we like may be incompatible with security. Or it may be years before we invent the reconciliation that allows us to revive some form of a feature we like without breaking security (as in my explanation of the Safra/Shapiro virtually nested virtual machines notion).
This then is the big dilemma. We have to have these features of exploration, even when the system gets secured (if you have the capabilities). So we need to implement the ownership and virtualization that you have talked about.
In the meantime, we have to break the feature, not security. If we proceed the other way around, we'll never be secure.
We wont? Why not? Given work on the environment to secure the stack, disable dynamic scoping, and providing for amplifiable inspectors/debuggers, we could support those features. Parallel work in the compiler and symbol spaces, could provide the language-based security we are talking about, but without losing the debuggability feature.
This means it'll go though a phase where it may be Smalltalk-like to many eyes, but where the cognoscenti may indeed judge that "it isn't going to be Smalltalk". If you're not willing to go through such a phase, don't bite off language-based security.
hmm. a dilemma.
Traits, continuations and my eventual sending redirectors are other examples where either dynamic scoping or system openness are crucial to allow us the facility.
Traits looks very cool, but I don't really understand them yet.
Allow me to correct myself. Traits looks *very* cool!
How do any of these require dynamic scoping or the existing kind of system openness (as opposed to the kind shown in Lex's Islands work)?
Traits had to subclass some of the Behavior classes to get this to work. Would that be possible in SqueakE? It has nothing to do with dynamic scoping.
The reason I include both dynamic scoping and system openness, is that the openness of Squeak _currently_ depends on dynamic scoping.
In what way? ParcPlace Smalltalk 2.5 and all ancestors I'm aware of going back to Smalltalk-76 (but not Smalltalk-72), had extreme system openness without any dynamic scoping.
I was under the assumption that these systems also walked the stack for exceptions and for debugging. This may not match the strict definition for dynamic scope, but it is openness that could go away without providing an alternative.
I read on your site about a new debugger, that includes a causality pane. Neat idea.
We can start this process by looking at providing more vm support for evental sending, finishing CapTP, and by migrating Lex's Islands work to 3.5.
Lex's Islands work defines a new language and core libraries that is in no way upwards compatible with existing Squeak. And it cannot, given his goals, and what I've been assuming are our goals.
That's an interesting observation. I don't necessarily think that changing the language is bad, and that might dictate a break in backwards compatibility. I am worried about loosing the environment and ability we have in the environment. Some of the discussion to date has informed me that it is possible to support those environment features, and although it is difficult work, it is work that we ought to address sooner than later, so that we always maintain a working environment.
In Lex' Islands work, he still had the debugger and inspector working, but has built a secure space. This is why I state that including Islands in the Interim step is important. It meets that security goal of isolating the space from other parts of the image. If it needs it's own Process, and disallows sharing of objects with other processes, then all the better.
Thus I think the short-term (couple of months) goal should be supporting security level 2 of Mark's original proposal.
I sent out that taxonomy in order to get a sense of how far the group was willing to go and at what price. The answer seemed to be, all the way (level 4). Since then, I've been posting on the assumption that that's what we're going for. Perhaps, now that the costs are clearer, a different level of ambition is desired. Well, that's one of the purposes of having a taxonomy.
Sure, the ambition may have changed in light of these costs. My main concern is we do this incrementally and preserve our ability to interact with the environment we are building as we go forward. This tells me that level 4 is still achievable, but we would be taking a long road to get there.
This is the only way I can think of that would encourage active participation. Without that, I can;t do it. I don;t have the knowledge, experience, or ability in many of these areas.
But all the levels in my earlier taxonomy require language-based security. (I failed to include the above "retreat" proposal in my earlier taxonomy.) Not even level 1 (Java applet-like: does no harm, but is mostly useless) can be achieved upwards compatibly, as Java does not: Applets effectively see a different library API than do Java applications. You cannot generally run Java applications as applets. If you could, there'd be no security.
I don't mean that protocols cannot change and tools evolve, when I say upwards compatible. It is an environment and tools argument.
I'm sorry fr not making that more clear.
Your right, I'm giving too much authority to Kernel creators. So instead I propose that once a creator "releases" his class he can no longer change it. If he wants to make revisions he has to create a new subclass. Others may decide to use his new subclass or not. They may migrate existing instances to the new class and/or use the new class for new instances.
I don't like this restriction. Classes and code should be able to evolve. Now if an 'Island' has classes bound into it, those classes shouldn't change, unless the right capabilities to change them are held.
And what are the right capabilities?
Write access into the meta-scope? It is something I could imagine would be disabled (revoked) for a production deployment.
I think the argument Mark is making is that there isn't a central authority. In fact, there is no third party security sever or third party interface respository. I do think that we could specify the protocols for a base library, but that's really what the ANSI spec is about. There may be some library specification additions for supporting security/eventual sends.
Well, I am making that point. But in the context of Anthony's design -- a distributed shared heap, in which all replicas of an object are considered to be the "same" object -- then, for replicas for which there's no one party that everyone else is willing to take as authoritative, such objects must be immutable. If you don't like the immutability, then some other assumption must give, which is plausible, but leads to a different design.
Interesting. This actually takes us back to the serialization thread. If you pass me a named structure (i.e type) then I should be free to decide into which object i map that structure. I can use any version of the Person class I want to as long as it's structure is compatible with the sent structure.
MarkM wrote:
Same problem then. I think these can't be automatically
replicated, because
there is no one authority everyone would mutually trust. At
least I hope there
won't be. A world in which there's some one authority
everyone is willing to
make themselves fully vulnerable to is a world ripe for
dictatorship.
Yes, like above I will change this so no one has authority once the pool is "released". However, I will still allow the creator to add new classes to a pool. This is where he could add his new subclass versions. Adding to a pool is benign.
Again, I think that with the right capability, these things could be changed 'locally'. The issue is that there isn't a global authority.
You're "locally" implies a change of assumptions that probably could be made to work, but breaks the uniformity of the distributed heap notion Anthony is trying for. It would lead to a different design.
That uniformity seems to require globally immutable classes. I don't think that this is realistic given the other flexible environment issues we have discussed.
I am encouraged that although this list is turning out to be on the quiet side, there is much interest in what we are considering. The fact that we are doing so in a thoughtful way is to our advantage.
I've been curious about the sudden quietness myself.
Well, this caused a little spark which is fantastic.
I will be a little more active this weekend, now that the work week is winding down. I want to implement RemoteHandlers (virtual circuits with remote objects) and the objectMap.
The other dimension, that I have been doing a little work with this past week, is cleaning up the EventualReferences. I am hanging out on the IRC channel #squeak-e and will be available there to guide anyone wanting to dive into this very intriguing mechanism. Look at ReferenceContext (the eventual reference), and RedirectionManager (the underlying redirection mechanism). If you build a redirection vm, make sure you doit to 'RedirectionManager enable'. I will update the SqueakElib package this evening.
Cool, looking forward to it!
Well, I have updated the package a few times. I needed to repair some bad integration breaks between eventual references, background secretaries (my Vat thread), and Morphic. They don't integrate well.
I hope this clarifies the delicate position I am espousing...
cheers! Rob