[Squeak-e] Comments on Anthony's "...Shared Smalltalk"

[Robert Withers]

On Saturday, February 8, 2003, at 02:02 PM, Mark S. Miller wrote:

> Rob, I admit I'm a bit confused by your message. Earlier, you wrote:
>

Mark, I'll admit that I am confused, myself.  I am confused about what 
the right way forward is, when contemplating the changes we need to 
make to reach level 4.

I still support getting to language-based security, rather than just 
framework-based, and I realize that this won't be possible without 
breaking things.

Joshua mentioned the failed modules effort.  Part of the reason it 
failed was because it changed some basic assumptions in the squeak 
environment and this made it unusable.  It left the community behind.  
The other concern is providing enough functionality so that the people 
here, for instance, may be able to start using it and making it better. 
  I would love it if that were to happen!

So I am not retreating, but trying to find the balance between having 
our environment and extending it with inabilities. :)   I would like to 
rename your 'A Coherent Retreat' proposal to 'A Functioning Interim 
Vat', since I am making good headway on CapTP.  It is the area where I 
can effect some benefit.

I personally am bound into an implementation perspective and I don't 
have knowledge or experience in the areas being discussed, around 
scoping and the compiler.  Hopefully my comments aren't a distraction 
to the discussion.  I also have some limited time in which to finish 
what I can, and hopefully incorporate it into squeak.  My limitation is 
that I am working full-time, while doing SqueakE work on the side.   In 
April, sailing season begins again and I expect that I will be on the 
water every weekend until October, so I won't have but a couple of 
hours a week to work on this through the summer, and that isn't the 
kind of time that would allow me to do deep squeaking.

This does not mean that I am opposed, in the long run, to the 
burn-the-diskpacks approach, only that it should be tempered with the 
realities of community building, therefore there are several milestones 
I am thinking in terms of.

First milestone is to see how integrated we can make the eventual 
sending mechanism into core squeak.  This is my goal for April.   This 
requires Andreas' ScriptProcessor, so that we can integrate eventual 
sending with the average user of Semaphores and sequential scripts.  
Andreas, if you have a more current version of your core scripting 
framework, I'd love to see it.

Second milestone is the virtualization layers to support debuggability.

Third is the compiler and language that would provide level 4 security.

There is no reason that they shouldn't be concurrent efforts, but we 
should point to that feature (system openness) as so important that it 
deserves its own milestone sooner rather than later.

On Saturday, February 8, 2003, at 06:08 PM, Joshua 'Schwa' Gargus wrote:

> I'm very
> interested in Squeak-E, and would like to incrementally modify my code
> to fix whatever breaks as APIs are tamed.  But I _need_ the
> programming environment that I am accustomed to; if Squeak-E can't
> support the debugger and other development tools, then I fear that the
> Squeak community will not follow.

This is the critical point when a burn-the-diskpacks approach is 
usable.  When it supports the investigation tools to allow us to 
manipulate it, it will be more broadly usable.  Anything we can do to 
continue working in an environment that currently supports these 
things, we should take advantage of.   This means an approach like 
Andreas has suggested seems required.  This approach will allow us to 
fundamentally change the language, while still working in our 
development environment.


On Saturday, February 8, 2003, at 02:02 PM, Mark S. Miller wrote:

> I think that, by doing this, we can eventually build back up to 
> something
> close to the level of functionality of current Squeak. But the result 
> will
> not be compatible. You cannot add inabilities to a system the way you 
> add a
> new package. New inabilities means that some old code must break.

When we do this, we should make every attempt to not "lock-down" the 
security so we can debug it with normal tools, until we can support a 
debugger and inspector of the right's amplification variety.  This 
tells me that one of the first things we do after getting CapTP  and 
Eventual Sending done, is to build your virtualization layers.  :)

We really have two orthogonal directions to take.  First is the 
execution environment and support for tools.  The other is the compiler 
and mutability and plugging leaks in the global scope.   If you agree 
that this seems to be a natural fault line in the work ahead, then 
perhaps we should form two sub-efforts.  The first can look at how to 
build an inspector and debugger, while the second look at the new 
Object hierarchy rooted at EObject and it's compiler.

> This process -- removing everything that doesn't follow capability
> discipline, followed by adding back in new APIs that provide similar
> functionality in a capability-compatible way -- is a kind of 
> refactoring,
> though a much more violent case then we normally encounter. A great 
> example
> is Lex's ObjectInspector in the Island's work. It moves the reflective
> messages from the objects themselves, where any client of the object 
> could
> invoke them, and puts them instead on ObjectInspector, which can be 
> given
> only to those that should indeed be authorized to break the 
> encapsulation of
> these objects. (The Brand system explained earlier enables the issue of
> "which objects" to be dealt with in a finer-grained fashion, but 
> that's not
> relevant to the points I make here.)

That's easy enough.  We have to change the MOP for our tools.  :)

I have been rereading your post on the KeyKOS brand and your comments 
that this could be placed in a register.  In other posts it almost 
seems that you are separating the lexical scoping from the code.   I am 
trying futilely to separate the different dimensions of an execution 
environment.  You have introduced the concept of ownership into the 
object graph and the idea that the scope is fixed at the time of 
creation of an object, rather than retrievable by walking the stack.  I 
can't begin to say that I understand this.

When the ideas are finally understood to the point where there is an 
implementation with an inspector and debugger, then we would have a 
usable environment.


> If you think you can get to capability security in an upwards 
> compatible
> way, what are you going to do about #instVarAt:? Or the APIs for 
> opening and
> writing any of the user's files? Do you really want any Squeaklet to 
> be able
> to do that?

Hopefully the MOP replacement for #instVarAt:, will allow us the same 
functionality.

> A Coherent Retreat
>
> There is another kind of capability security you can achieve with full
> upwards compatibility. It is indeed quite useful, it is the one you've
> already made wonderful progress on, it is where E started and stayed 
> for
> years, it is the way E's largest customer is using it, and it is also 
> the
> kind of security supported by Waterken Software 
> http://www.waterken.com . If
> indeed the Squeak community is not willing to take the steps needed to
> design and build a Squeak-like capability secure *language*, then this 
> may
> indeed be a good place to retreat to. In any case, it's something you 
> need
> to build anyway, it's something you are building anyway, and it may be 
> a
> good place to start.

The fact that this is 1) valuable and 2) something we need anyway, 
tells me that we ought to make it a first step on the way to the 
broader changes.  That we may be able to do this and maintain mostly 
backward compatibility is encouraging to me.


> What you give up is language-based security. Nothing above leverages 
> the
> almost-capability nature of Smalltalk. You wouldn't be achieving 
> anything
> that couldn't be achieved equally well by C++ programmers using a CapTP
> library written in C++. You'd have no ability to locally run code you 
> don't
> trust, and therefore no ability to make Squeaklets happen.

This is still a goal, but we need to preserve our environment to get 
there.


> So if this is indeed done as a retreat, it would sadden me. I think 
> that a
> Squeak-E *language* would be a wonderful thing.

I think it would be too, but it is a long way off.  As we strive to get 
there, let's make sure we keep it a running system.



> In the kind of massive refactoring we'd need to get language-based 
> security,
> not only can't we be upwards compatible, we can't necessarily retain 
> all the
> features we'd like. Our likes evolved without constraint of security. 
> Some
> things we like may be incompatible with security. Or it may be years 
> before
> we invent the reconciliation that allows us to revive some form of a 
> feature
> we like without breaking security (as in my explanation of the 
> Safra/Shapiro
> virtually nested virtual machines notion).

This then is the big dilemma.  We have to have these features of 
exploration, even when the system gets secured (if you have the 
capabilities).   So we need to implement the ownership and 
virtualization that you have talked about.


> In the meantime, we have to break the feature, not security. If we 
> proceed
> the other way around, we'll never be secure.

We wont?  Why not?   Given work on the environment to secure the stack, 
disable dynamic scoping, and providing for amplifiable 
inspectors/debuggers, we could support those features.  Parallel work 
in the compiler and symbol spaces, could provide the language-based 
security we are talking about, but without losing the debuggability 
feature.


> This means it'll go though a
> phase where it may be Smalltalk-like to many eyes, but where the 
> cognoscenti
> may indeed judge that "it isn't going to be Smalltalk". If you're not
> willing to go through such a phase, don't bite off language-based 
> security.

hmm.  a dilemma.



>> Traits, continuations and my eventual sending redirectors are other 
>> examples
>> where either dynamic scoping or system openness are crucial to allow 
>> us the
>> facility.
>
> Traits looks very cool, but I don't really understand them yet.
>
> Allow me to correct myself. Traits looks *very* cool!
>
> How do any of these require dynamic scoping or the existing kind of 
> system
> openness (as opposed to the kind shown in Lex's Islands work)?

Traits had to subclass some of the Behavior classes to get this to 
work.  Would that be possible in SqueakE?  It has nothing to do with 
dynamic scoping.


>>  The reason I include both dynamic scoping and system openness, is
>> that the openness of Squeak _currently_ depends on dynamic scoping.
>
> In what way? ParcPlace Smalltalk 2.5 and all ancestors I'm aware of 
> going
> back to Smalltalk-76 (but not Smalltalk-72), had extreme system 
> openness
> without any dynamic scoping.

I was under the assumption that these systems also walked the stack for 
exceptions and for debugging.  This may not match the strict definition 
for dynamic scope, but it is openness that could go away without 
providing an alternative.

I read on your site about a new debugger, that includes a causality 
pane.   Neat idea.


>> We can start this process by looking at providing more vm support for
>> evental sending, finishing CapTP, and by migrating Lex's Islands work 
>> to
>> 3.5.
>
> Lex's Islands work defines a new language and core libraries that is 
> in no
> way upwards compatible with existing Squeak. And it cannot, given his 
> goals,
> and what I've been assuming are our goals.


That's an interesting observation.  I don't necessarily think that 
changing the language is bad, and that might dictate a break in 
backwards compatibility.   I am worried about loosing the environment 
and ability we have in the environment.  Some of the discussion to date 
has informed me that it is possible to support those environment 
features, and although it is difficult work, it is work that we ought 
to address sooner than later, so that we always maintain a working 
environment.

In Lex' Islands work, he still had the debugger and inspector working, 
but has built a secure space.  This is why I state that including 
Islands in the Interim step is important.  It meets that security goal 
of isolating the space from other parts of the image.  If it needs it's 
own Process, and disallows sharing of objects with other processes, 
then all the better.


>> Thus I think the short-term (couple of months) goal should be
>> supporting security level 2 of Mark's original proposal.
>
> I sent out that taxonomy in order to get a sense of how far the group 
> was
> willing to go and at what price. The answer seemed to be, all the way
> (level 4). Since then, I've been posting on the assumption that that's 
> what
> we're going for. Perhaps, now that the costs are clearer, a different 
> level
> of ambition is desired. Well, that's one of the purposes of having a 
> taxonomy.

Sure, the ambition may have changed in light of these costs.  My main 
concern is we do this incrementally and preserve our ability to 
interact with the environment we are building as we go forward.  This 
tells me that level 4 is still achievable, but we would be taking a 
long road to get there.

This is the only way I can think of that would encourage active 
participation.   Without that, I can;t do it.  I don;t have the 
knowledge, experience, or ability in many of these areas.


> But all the levels in my earlier taxonomy require language-based 
> security.
> (I failed to include the above "retreat" proposal in my earlier 
> taxonomy.)
> Not even level 1 (Java applet-like: does no harm, but is mostly 
> useless) can
> be achieved upwards compatibly, as Java does not: Applets effectively 
> see a
> different library API than do Java applications. You cannot generally 
> run
> Java applications as applets. If you could, there'd be no security.

I don't mean that protocols cannot change and tools evolve, when I say 
upwards compatible.  It is an environment and tools argument.

I'm sorry fr not making that more clear.


>>> Your right, I'm giving too much authority to Kernel creators.
>>>  So instead
>>> I propose that once a creator "releases" his class he can no longer
>>> change it.  If he wants to make revisions he has to create a new
>>> subclass.  Others may decide to use his new subclass or not.  They 
>>> may
>>> migrate existing instances to the new class and/or use the
>>> new class for
>>> new instances.
>>
>> I don't like this restriction.  Classes and code should be able to 
>> evolve.
>> Now if an 'Island' has classes bound into it, those classes shouldn't
>> change, unless the right capabilities to change them are held.
>
> And what are the right capabilities?

Write access into the meta-scope?  It is something I could imagine 
would be disabled (revoked) for a production deployment.


>> I think the argument Mark is making is that there isn't a central 
>> authority.
>> In fact, there is no third party security sever or third party 
>> interface
>> respository.  I do think that we could specify the protocols for a 
>> base
>> library, but that's really what the ANSI spec is about.  There may be 
>> some
>> library specification additions for supporting security/eventual 
>> sends.
>
> Well, I am making that point. But in the context of Anthony's design 
> -- a
> distributed shared heap, in which all replicas of an object are 
> considered
> to be the "same" object -- then, for replicas for which there's no one 
> party
> that everyone else is willing to take as authoritative, such objects 
> must be
> immutable. If you don't like the immutability, then some other 
> assumption
> must give, which is plausible, but leads to a different design.

Interesting.  This actually takes us back to the serialization thread.  
If you pass me a named structure (i.e type) then I should be free to 
decide into which object i map that structure.  I can use any version 
of the Person class I want to as long as it's structure is compatible 
with the sent structure.


>>> MarkM wrote:
>>
>>>> Same problem then. I think these can't be automatically
>>> replicated, because
>>>> there is no one authority everyone would mutually trust. At
>>> least I hope there
>>>> won't be. A world in which there's some one authority
>>> everyone is willing to
>>>> make themselves fully vulnerable to is a world ripe for
>>> dictatorship.
>>>
>>> Yes, like above I will change this so no one has authority once the
>>> pool is "released".  However, I will still allow the creator
>>> to add new
>>> classes to a pool.  This is where he could add his new
>>> subclass versions.
>>> Adding to a pool is benign.
>>
>> Again, I think that with the right capability, these things could be 
>> changed
>> 'locally'.  The issue is that there isn't a global authority.
>
> You're "locally" implies a change of assumptions that probably could 
> be made
> to work, but breaks the uniformity of the distributed heap notion 
> Anthony is
> trying for. It would lead to a different design.

That uniformity seems to require globally immutable classes.  I don't 
think that this is realistic given the other flexible environment 
issues we have discussed.


>> I am encouraged that although this list is turning out to be on the 
>> quiet
>> side, there is much interest in what we are considering.  The fact 
>> that we
>> are doing so in a thoughtful way is to our advantage.
>
> I've been curious about the sudden quietness myself.

Well, this caused a little spark which is fantastic.


>> I will be a little more active this weekend, now that the work week is
>> winding down.  I want to implement RemoteHandlers (virtual circuits 
>> with
>> remote objects) and the objectMap.
>>
>> The other dimension, that I have been doing a little work with this 
>> past
>> week, is cleaning up the EventualReferences.   I am hanging out on 
>> the IRC
>> channel #squeak-e and will be available there to guide anyone wanting 
>> to
>> dive into this very intriguing mechanism.  Look at ReferenceContext 
>> (the
>> eventual reference), and RedirectionManager (the underlying 
>> redirection
>> mechanism).  If you build a redirection vm, make sure you doit to
>> 'RedirectionManager enable'.  I will update the SqueakElib package 
>> this
>> evening.
>
> Cool, looking forward to it!

Well, I have updated the package a few times.  I needed to repair some 
bad integration breaks between eventual references, background 
secretaries (my Vat thread), and Morphic.  They don't integrate well.

I hope this clarifies the delicate position I am espousing...

cheers!
Rob