Hi all.
Does anybody have code (particularly VM modifications) which allow Capabilities in Squeak?
In particular, I'm referring to code that implements stuff described on this page: http://minnow.cc.gatech.edu/squeak/uploads/2074/sandbox.html
In the past there have been a couple of ambitious projects that allow security in Squeak such as Squeak-E. What became of these? Is there code floating around somewhere? I'm only seeing Squeak-ELib on squeakmap.
I'm particularly interested in a debugger which uses Capabilities to debug/inspect objects rather than invoke methods such as Object>>printOn:. I'm working with message capture stuff, and the debugger falls over badly.
Michael.
I've been using a class called "MessageCapture" which lets you receive messages sent to an object. It's caused me no end of agony trying to get it working, so I thought I'd share my agony with the community :-P.
Source code here: http://www.squeaksource.com/DPON.html - click on "latest", MessageCapture-mvdg.2.mcz.
It's released under the.. umm... MIT license.
Also included is a Future class, which lets you do things like:
result := Future doing: [ some long computation ].
This will return immediately, forking off a process to do the computation and replacing "result" with the result when it's finished.
Some of the tests fail. Any hints / tips / guidance is very welcome! Especially as to how to capture #== and #class. Those two messages are sent by the interpreter even before doing a message lookup.
Michael.
Am 15.10.2006 um 08:56 schrieb Michael van der Gulik:
I've been using a class called "MessageCapture" which lets you receive messages sent to an object. It's caused me no end of agony trying to get it working, so I thought I'd share my agony with the community :-P.
Source code here: http://www.squeaksource.com/DPON.html - click on "latest", MessageCapture-mvdg.2.mcz.
It's released under the.. umm... MIT license.
Also included is a Future class, which lets you do things like:
result := Future doing: [ some long computation ].
This will return immediately, forking off a process to do the computation and replacing "result" with the result when it's finished.
Some of the tests fail. Any hints / tips / guidance is very welcome! Especially as to how to capture #== and #class. Those two messages are sent by the interpreter even before doing a message lookup.
You might want to look at how this is handled in Croquet (MIT licensed, too). As you know (or maybe not) Croquet is not foremost about 3D but about distributed computing with capability-based security. The messaging infrastructure with future sends, returned promises etc. is independent of the UI.
Now the current implementation does not yet provide water-tight security, but the principles are there and it's surely worth having a look.
- Bert -
Bert Freudenberg wrote:
Am 15.10.2006 um 08:56 schrieb Michael van der Gulik:
I've been using a class called "MessageCapture" which lets you receive messages sent to an object. It's caused me no end of agony trying to get it working, so I thought I'd share my agony with the community :-P.
Source code here: http://www.squeaksource.com/DPON.html - click on "latest", MessageCapture-mvdg.2.mcz.
It's released under the.. umm... MIT license.
Also included is a Future class, which lets you do things like:
result := Future doing: [ some long computation ].
This will return immediately, forking off a process to do the computation and replacing "result" with the result when it's finished.
Some of the tests fail. Any hints / tips / guidance is very welcome! Especially as to how to capture #== and #class. Those two messages are sent by the interpreter even before doing a message lookup.
You might want to look at how this is handled in Croquet (MIT licensed, too). As you know (or maybe not) Croquet is not foremost about 3D but about distributed computing with capability-based security. The messaging infrastructure with future sends, returned promises etc. is independent of the UI.
Now the current implementation does not yet provide water-tight security, but the principles are there and it's surely worth having a look.
Correct me if I'm wrong, but I think Croquet uses pretty normal objects for what it does. Objects are grouped together, and inter-group references are done using FarRef objects. I think it completely avoids the problem of capturing references - arguably a more stable approach than what I'm doing.
Can anybody confirm this?
I'm aware that Spoon and Magma also do message capture. I like Spoon's approach best: modify the VM.
Michael.
On Oct 16, 2006, at 4:32 AM, Michael van der Gulik wrote:
Bert Freudenberg wrote:
Am 15.10.2006 um 08:56 schrieb Michael van der Gulik:
I've been using a class called "MessageCapture" which lets you receive messages sent to an object. It's caused me no end of agony trying to get it working, so I thought I'd share my agony with the community :-P.
Source code here: http://www.squeaksource.com/DPON.html - click on "latest", MessageCapture-mvdg.2.mcz.
It's released under the.. umm... MIT license.
Also included is a Future class, which lets you do things like:
result := Future doing: [ some long computation ].
This will return immediately, forking off a process to do the computation and replacing "result" with the result when it's finished.
Some of the tests fail. Any hints / tips / guidance is very welcome! Especially as to how to capture #== and #class. Those two messages are sent by the interpreter even before doing a message lookup.
You might want to look at how this is handled in Croquet (MIT licensed, too). As you know (or maybe not) Croquet is not foremost about 3D but about distributed computing with capability- based security. The messaging infrastructure with future sends, returned promises etc. is independent of the UI. Now the current implementation does not yet provide water-tight security, but the principles are there and it's surely worth having a look.
Correct me if I'm wrong, but I think Croquet uses pretty normal objects for what it does. Objects are grouped together, and inter- group references are done using FarRef objects.
That's correct. Croquet also has a notion of "future", but unlike your Future example above, there is no immediately-returned placeholder that is later replaced (via #become: ?) by the real value when it becomes known. Instead, the immediately-returned value is another FarRef. To determine whether the computation has finished, you can ask it #isResolved, or you can use 'aFarRef wait' to block until the FarRef is resolved.
Josh
I think it completely avoids the problem of capturing references - arguably a more stable approach than what I'm doing.
Can anybody confirm this?
I'm aware that Spoon and Magma also do message capture. I like Spoon's approach best: modify the VM.
Michael.
Am 16.10.2006 um 10:32 schrieb Michael van der Gulik:
Bert Freudenberg wrote:
Am 15.10.2006 um 08:56 schrieb Michael van der Gulik:
I've been using a class called "MessageCapture" which lets you receive messages sent to an object. It's caused me no end of agony trying to get it working, so I thought I'd share my agony with the community :-P.
Source code here: http://www.squeaksource.com/DPON.html - click on "latest", MessageCapture-mvdg.2.mcz.
It's released under the.. umm... MIT license.
Also included is a Future class, which lets you do things like:
result := Future doing: [ some long computation ].
This will return immediately, forking off a process to do the computation and replacing "result" with the result when it's finished.
Some of the tests fail. Any hints / tips / guidance is very welcome! Especially as to how to capture #== and #class. Those two messages are sent by the interpreter even before doing a message lookup.
You might want to look at how this is handled in Croquet (MIT licensed, too). As you know (or maybe not) Croquet is not foremost about 3D but about distributed computing with capability- based security. The messaging infrastructure with future sends, returned promises etc. is independent of the UI. Now the current implementation does not yet provide water-tight security, but the principles are there and it's surely worth having a look.
Correct me if I'm wrong, but I think Croquet uses pretty normal objects for what it does. Objects are grouped together, and inter- group references are done using FarRef objects. I think it completely avoids the problem of capturing references - arguably a more stable approach than what I'm doing.
Can anybody confirm this?
In the current implementation, yes. An earlier version used many automagic tricks to work transparently. But Magic Is Hard To Debug (TM). So I think the plan is to do it with normal objects first, do everything explicitly, get it right, and when the system really works, then you can add back the magic to make it easier to use. But broken magic does more harm than good.
- Bert -
Bert Freudenberg wrote:
Am 16.10.2006 um 10:32 schrieb Michael van der Gulik:
Correct me if I'm wrong, but I think Croquet uses pretty normal objects for what it does. Objects are grouped together, and inter- group references are done using FarRef objects. I think it completely avoids the problem of capturing references - arguably a more stable approach than what I'm doing.
Can anybody confirm this?
In the current implementation, yes. An earlier version used many automagic tricks to work transparently. But Magic Is Hard To Debug (TM). So I think the plan is to do it with normal objects first, do everything explicitly, get it right, and when the system really works, then you can add back the magic to make it easier to use. But broken magic does more harm than good.
They're very smart. I've pushed along with broken magic for a couple of years. Believe me - it would be much, much easier not using broken magic.
Currently I'm trying to debug an image which freezes at random intervals. Alt-. is useless and I've already applied my alt-. improving tweaks. I've inserted print statements everywhere - none of them get printed. Broken magic is fun :-#.
Michael.
Hi Michael,
how would you execute [ some long computation ] on another processor or another computer.
On another processor, Squeak does't work because its VM is neither reentrant nor multiprocessing-capable (sorry for using terms which are > 30 years old ;-)
So it would be interesting to learn how you plan to offload [ some long computation ] to another computer (or, FWIW, to another process running on the same computer).
/Klaus
On Sun, 15 Oct 2006 08:56:48 +0200, Michael wrote:
I've been using a class called "MessageCapture" which lets you receive messages sent to an object. It's caused me no end of agony trying to get it working, so I thought I'd share my agony with the community :-P.
Source code here: http://www.squeaksource.com/DPON.html - click on "latest", MessageCapture-mvdg.2.mcz.
It's released under the.. umm... MIT license.
Also included is a Future class, which lets you do things like:
result := Future doing: [ some long computation ].
This will return immediately, forking off a process to do the computation and replacing "result" with the result when it's finished.
Some of the tests fail. Any hints / tips / guidance is very welcome! Especially as to how to capture #== and #class. Those two messages are sent by the interpreter even before doing a message lookup.
Michael.
Actually, you could fork the VM, establish a communication channel (I've used named pipes on Unix, works without VM modification), send the future message over there, wait for the result. As long as no full GC happens in either process, this is even memory efficient on systems using copy-on-write memory pages.
Offloading to a different machine is a bit more involved because you might have to clone the execution state in the master image first.
- Bert -
Am 15.10.2006 um 14:46 schrieb Klaus D. Witzel:
Hi Michael,
how would you execute [ some long computation ] on another processor or another computer.
On another processor, Squeak does't work because its VM is neither reentrant nor multiprocessing-capable (sorry for using terms which are > 30 years old ;-)
So it would be interesting to learn how you plan to offload [ some long computation ] to another computer (or, FWIW, to another process running on the same computer).
/Klaus
On Sun, 15 Oct 2006 08:56:48 +0200, Michael wrote:
I've been using a class called "MessageCapture" which lets you receive messages sent to an object. It's caused me no end of agony trying to get it working, so I thought I'd share my agony with the community :-P.
Source code here: http://www.squeaksource.com/DPON.html - click on "latest", MessageCapture-mvdg.2.mcz.
It's released under the.. umm... MIT license.
Also included is a Future class, which lets you do things like:
result := Future doing: [ some long computation ].
This will return immediately, forking off a process to do the computation and replacing "result" with the result when it's finished.
Some of the tests fail. Any hints / tips / guidance is very welcome! Especially as to how to capture #== and #class. Those two messages are sent by the interpreter even before doing a message lookup.
Michael.
On Sun, 15 Oct 2006 14:56:38 +0200, Bert Freudenberg wrote:
Actually, you could fork the VM, establish a communication channel (I've used named pipes on Unix, works without VM modification), send the future message over there, wait for the result. As long as no full GC happens in either process, this is even memory efficient on systems using copy-on-write memory pages.
Aha, this is how to avoid serialization of context(s), method(s) and other resistant factors.
Offloading to a different machine is a bit more involved because you might have to clone the execution state in the master image first.
I agree. So whatever the desired path to the solution, doesn't serialization ("clone the execution state") presuppose that at least the compiled methods are in sync between any pair of involved systems.
/Klaus
- Bert -
Am 15.10.2006 um 14:46 schrieb Klaus D. Witzel:
Hi Michael,
how would you execute [ some long computation ] on another processor or another computer.
On another processor, Squeak does't work because its VM is neither reentrant nor multiprocessing-capable (sorry for using terms which are
30 years old ;-)
So it would be interesting to learn how you plan to offload [ some long computation ] to another computer (or, FWIW, to another process running on the same computer).
/Klaus
On Sun, 15 Oct 2006 08:56:48 +0200, Michael wrote:
I've been using a class called "MessageCapture" which lets you receive messages sent to an object. It's caused me no end of agony trying to get it working, so I thought I'd share my agony with the community :-P.
Source code here: http://www.squeaksource.com/DPON.html - click on "latest", MessageCapture-mvdg.2.mcz.
It's released under the.. umm... MIT license.
Also included is a Future class, which lets you do things like:
result := Future doing: [ some long computation ].
This will return immediately, forking off a process to do the computation and replacing "result" with the result when it's finished.
Some of the tests fail. Any hints / tips / guidance is very welcome! Especially as to how to capture #== and #class. Those two messages are sent by the interpreter even before doing a message lookup.
Michael.
On Sun, Oct 15, 2006 at 02:56:38PM +0200, Bert Freudenberg wrote:
Actually, you could fork the VM, establish a communication channel (I've used named pipes on Unix, works without VM modification), send the future message over there, wait for the result. As long as no full GC happens in either process, this is even memory efficient on systems using copy-on-write memory pages.
Why the named pipe? An OSPipe should work fine without requiring any external setup outside of Squeak.
Dave
Am 15.10.2006 um 19:08 schrieb David T. Lewis:
On Sun, Oct 15, 2006 at 02:56:38PM +0200, Bert Freudenberg wrote:
Actually, you could fork the VM, establish a communication channel (I've used named pipes on Unix, works without VM modification), send the future message over there, wait for the result. As long as no full GC happens in either process, this is even memory efficient on systems using copy-on-write memory pages.
Why the named pipe? An OSPipe should work fine without requiring any external setup outside of Squeak.
The project I use the named pipes in does not have the OSProcessPlugin. But sure, if you use OSProcess for forking, OSPipes are available :-)
- Bert -
Klaus D. Witzel wrote:
Hi Michael,
how would you execute [ some long computation ] on another processor or another computer.
Simple. To use another processor, just modify the VM to use pthreads. Easy peasy! (*)
To do it on another computer, use my DPON project. Unfortunately it's current status is "completely freezes the Squeak image after 5 seconds". That's a bit hard to debug.
(*) this is sarcasm.
On another processor, Squeak does't work because its VM is neither reentrant nor multiprocessing-capable (sorry for using terms which are
30 years old ;-)
One day, after I've finished all my other projects, I'll be looking at making the Squeak VM use pthreads so it can use multiple CPUs. Hopefully somebody will beat me to it :-).
Michael.
On Mon, 16 Oct 2006 10:37:52 +0200, Michael van der Gulik wrote:
Klaus D. Witzel wrote:
Hi Michael, how would you execute [ some long computation ] on another processor or another computer.
Simple. To use another processor, just modify the VM to use pthreads. Easy peasy! (*)
:)
To do it on another computer, use my DPON project. Unfortunately it's current status is "completely freezes the Squeak image after 5 seconds". That's a bit hard to debug.
:(
(*) this is sarcasm.
Sure 8-)
On another processor, Squeak does't work because its VM is neither reentrant nor multiprocessing-capable (sorry for using terms which are
30 years old ;-)
One day, after I've finished all my other projects, I'll be looking at making the Squeak VM use pthreads so it can use multiple CPUs. Hopefully somebody will beat me to it :-).
Hopefully :)
/Klaus
Michael.
On Oct 14, 2006, at 9:45 PM, Michael van der Gulik wrote:
In the past there have been a couple of ambitious projects that allow security in Squeak such as Squeak-E. What became of these? Is there code floating around somewhere? I'm only seeing Squeak-ELib on squeakmap.
Squeak-Elib is the only result from Squeak-E. I also implement a Future class (called a Promise) which accepts messages and forwards them prior to resolving to a value. It fails in 2 senses.
First of all there are bugs when resolving the result of a computation to a promise, particularly when going inter-vat, which is handled differently than an intra-vat promise.
Secondly, FarRefs and promises don't understand all the base protocol that a normal object understands so many of the tools in the image don't deal well with eventual objects. Ultimately, this is your issue with needing VM changes, I believe. Primitives cannot handle eventual arguments. One example is when sending a #printString to an eventual object returns a promise then is used as the argument for rendering in Morphic. Another example is an eventual ref to true is sent the message #ifTrue:. What I think you would want to have is to protect the primitives. In a primitive, if an argument is eventual, send an "invokePrimitive" message to that ref, such that the primitive will be invoked eventually. That changes all primitives, unfortunately.
I have a vm changes file that changes #== and #class out there.
I have stopped development of Squeak-Elib.
I hope this helps, Robert
Robert Withers wrote:
Secondly, FarRefs and promises don't understand all the base protocol that a normal object understands so many of the tools in the image don't deal well with eventual objects.
Actually, I consider this a fatal bug of FarRefs which I finally solved in the Croquet version (TFarRefs). In Croquet, we do not hide the difference between "near" and "far" objects simply because we cannot hide the difference in the transport protocol utilized. Since all replicated messages require a roundtrip to the router before they are executed you can kiss the idea of a "transparent" handling goodbye, even with promise pipelining and all the other tricks. It's simply not possible and trying to hide the difference makes the problems much worse than admitting the differences and giving people a way to deal with it. One of the things that gets relevant in a hurry for example is that when you're behind a DSL link your upstream bandwidth is *extremely* limited and you simply need to know which messages you send where and when and why upstream. And that means the programmer needs to see when things go over the wire, these messages simply must be explicit. [My apologies go to both MarkM and MarcS who consistently told me that this would be the case but until I had the chance of looking at "other people's code" I hadn't realized how big both the conceptual as well as the practical mess is if you try to "make those differences go away"].
So, today when you use Croquet and have a far ref you *must* send explicit future messages. TFarRef itself is a plain old Object subclass which means printing, inspecting and debugging works just fine. And I am in the process of doing precisely the same for Tweak which will dramatically simplify a lot of stuff that was just unbelievably awkward before.
Cheers, - Andreas
On Oct 16, 2006, at 8:42 PM, Andreas Raab wrote:
Robert Withers wrote:
Secondly, FarRefs and promises don't understand all the base protocol that a normal object understands so many of the tools in the image don't deal well with eventual objects.
Actually, I consider this a fatal bug of FarRefs which I finally solved in the Croquet version (TFarRefs).
I think that's pretty smart. My description of changing the primitives to be eventual aware are intended to describe my concept of going the other way and that means that any object could possibly be eventual. Whether they are remote or not doesn't matter - it truly is a change in the execution semantics of the VM and it's best to make that change rather than doing what I was doing. Of course, this doesn't address issues that may arise due to latency or ordering which could still affect all those tools. I agree that you may still desire to be more explicit when dealing with remote objects, but to my way of thinking that is secondary to the idea of making the VM eventual. Anyway, that is how you would need to do it to get SqueakElib truly working in the image.
I think it's smart to have done what you did.
Cheers, Robert
I think it's smart to have done what you did.
Thanks. Now if I could only claim that to be my idea... ;-)
Cheers, - Andreas
Robert Withers wrote:
On Oct 16, 2006, at 8:42 PM, Andreas Raab wrote:
Robert Withers wrote:
Secondly, FarRefs and promises don't understand all the base protocol that a normal object understands so many of the tools in the image don't deal well with eventual objects.
Actually, I consider this a fatal bug of FarRefs which I finally solved in the Croquet version (TFarRefs).
I think that's pretty smart. My description of changing the primitives to be eventual aware are intended to describe my concept of going the other way and that means that any object could possibly be eventual. Whether they are remote or not doesn't matter - it truly is a change in the execution semantics of the VM and it's best to make that change rather than doing what I was doing. Of course, this doesn't address issues that may arise due to latency or ordering which could still affect all those tools. I agree that you may still desire to be more explicit when dealing with remote objects, but to my way of thinking that is secondary to the idea of making the VM eventual. Anyway, that is how you would need to do it to get SqueakElib truly working in the image.
I think it's smart to have done what you did.
Cheers, Robert
On Oct 16, 2006, at 9:47 PM, Andreas Raab wrote:
I think it's smart to have done what you did.
Thanks. Now if I could only claim that to be my idea... ;-)
ditto. :-) I can't recall where those ideas I expressed originated, perhaps yourself, but I was expressing them as opinions.
Cheers, Robert
Robert Withers wrote:
On Oct 16, 2006, at 8:42 PM, Andreas Raab wrote:
Robert Withers wrote:
Secondly, FarRefs and promises don't understand all the base protocol that a normal object understands so many of the tools in the image don't deal well with eventual objects.
Actually, I consider this a fatal bug of FarRefs which I finally solved in the Croquet version (TFarRefs).
I think that's pretty smart. My description of changing the primitives to be eventual aware are intended to describe my concept of going the other way and that means that any object could possibly be eventual. Whether they are remote or not doesn't matter - it truly is a change in the execution semantics of the VM and it's best to make that change rather than doing what I was doing. Of course, this doesn't address issues that may arise due to latency or ordering which could still affect all those tools. I agree that you may still desire to be more explicit when dealing with remote objects, but to my way of thinking that is secondary to the idea of making the VM eventual. Anyway, that is how you would need to do it to get SqueakElib truly working in the image. I think it's smart to have done what you did. Cheers, Robert
Robert Withers wrote:
On Oct 14, 2006, at 9:45 PM, Michael van der Gulik wrote:
In the past there have been a couple of ambitious projects that allow security in Squeak such as Squeak-E. What became of these? Is there code floating around somewhere? I'm only seeing Squeak-ELib on squeakmap.
Squeak-Elib is the only result from Squeak-E. I also implement a Future class (called a Promise) which accepts messages and forwards them prior to resolving to a value. It fails in 2 senses.
First of all there are bugs when resolving the result of a computation to a promise, particularly when going inter-vat, which is handled differently than an intra-vat promise.
Secondly, FarRefs and promises don't understand all the base protocol that a normal object understands so many of the tools in the image don't deal well with eventual objects.
Yea, I've discovered that. My next project will be modifying the debugger to not send *any* messages to the objects it's debugging, but rather use a capability with primitives to peek at each object's state.
Ultimately, this is your issue with needing VM changes, I believe. Primitives cannot handle eventual arguments. One example is when sending a #printString to an eventual object returns a promise then is used as the argument for rendering in Morphic. Another example is an eventual ref to true is sent the message #ifTrue:. What I think you would want to have is to protect the primitives. In a primitive, if an argument is eventual, send an "invokePrimitive" message to that ref, such that the primitive will be invoked eventually. That changes all primitives, unfortunately.
I've solved this (with many bugs and instabilities) by making Futures block on any message sends until the result returns.
The way I've done it is that MessageCapture is a subclass of ProtoObject that overrides #doesNotUnderstand. I added a few methods like #printString to make it work with the debugger and added a MethodCaptureInspecter that will let you inspect a MessageCapture.
The Future itself is a normal object which uses MessageCapture.
This works for primitive methods, but there are issues with some sends which I think I can solve. I think Craig Latta's approach in Spoon is the best: create a special class (MessageCapture) which the VM checks for on every message send, including #== and #class. If it is that class, package up a Message and send it to a method on that class.
I have a vm changes file that changes #== and #class out there.
Here?: http://minnow.cc.gatech.edu/squeak/2410
I'll be making changes to the VM eventually, so this code is much appreciated. Do you mind if I consider it released under the MIT license?
Michael.
On Oct 17, 2006, at 12:19 AM, Michael van der Gulik wrote:
I have a vm changes file that changes #== and #class out there.
Here's a better link: http://www.squeaksource.com/squeakelib/ SqueakElibVM-rww.1.mcz
I'll be making changes to the VM eventually, so this code is much appreciated. Do you mind if I consider it released under the MIT license?
Yes, I released under MIT.
Cheers, Robert
On 10/18/06, Robert Withers reefedjib@yahoo.com wrote:
On Oct 17, 2006, at 12:19 AM, Michael van der Gulik wrote:
I have a vm changes file that changes #== and #class out there.
Here's a better link: http://www.squeaksource.com/squeakelib/ SqueakElibVM-rww.1.mcz
That's a very small .mcz :-). I've just had a look at your changes, and while they'd work, they'd come with a speed penalty. Essentially, you're just un-doing the optimisations on #== and #class.
I'm not sure this is the right way to go. I'll think about it some more.
Thanks for the code in any case!
Michael.
On Oct 17, 2006, at 4:52 PM, Michael van der Gulik wrote:
On 10/18/06, Robert Withers reefedjib@yahoo.com wrote:
On Oct 17, 2006, at 12:19 AM, Michael van der Gulik wrote:
I have a vm changes file that changes #== and #class out there.
Here's a better link: http://www.squeaksource.com/squeakelib/ SqueakElibVM-rww.1.mcz
That's a very small .mcz :-). I've just had a look at your changes, and while they'd work, they'd come with a speed penalty. Essentially, you're just un-doing the optimisations on #== and #class.
Yes, I did it solely to achieve the functional benefits when working with SqueakElib. Identity is an interesting issue when dealing with eventual refs.
I'm not sure this is the right way to go. I'll think about it some more.
Fair enough.
cheers, Robert
That's a very small .mcz :-). I've just had a look at your changes, and while they'd work, they'd come with a speed penalty. Essentially, you're just un-doing the optimisations on #== and #class.
Yes, I did it solely to achieve the functional benefits when working with SqueakElib. Identity is an interesting issue when dealing with eventual refs.
Islands also required undoing some of these optimizations. Especially #class needed to do something non-trivial, because Squeak's normal #class lets people redefine the class hierarchy and even the language!
I would not rush to assume there is a real performane hit, especially for #class. For #==, maybe, but remember that a sophisticated #== is a fundamental part of Elib's design.
-Lex
Michael van der Gulik squeakml@gulik.co.nz writes:
Hi all.
Does anybody have code (particularly VM modifications) which allow Capabilities in Squeak?
In particular, I'm referring to code that implements stuff described on this page: http://minnow.cc.gatech.edu/squeak/uploads/2074/sandbox.html
This is my old "Islands" project, done during an summer internship with the Squeak group in 2000. The group was just starting to work on sharing EToy's. For example, that was when the SuperSwiki and the web-browser plugin were developed.
I had recently been "Millered", and was fascinated by the possibility of object references and message sending being your security kernel. So, I attempted to use this approach so that you could run untrusted EToy's inside Squeak without needing a special VM mode. You could then email someone an EToy and they could read play with it right in Celeste.
I got so far as getting BouncingAtomsMorph to run in a sandbox, but then ran out of time. I went back to hammering on type inference, which was engrossing and sucked away all of my next 6 years.
Full information about Islands is available at this page:
http://minnow.cc.gatech.edu/squeak/2074
It includes the rationale, the source code, a pre-built image, and notes about updating it to newer Squeaks.
I still think the basic approach is good. The next thing I would do, were I to continue, would be to get rid of the dynamically bound global variables, and instead to have separate, static namespaces. The reason for the current approach--i.e. all global references are bound indirectly through the currently active island--is that compiled code can be reused across multiple islands. In retrospect, it would be better to maintain conceptual pruity and simply recompile any reused code.
More broadly, I still think the object capabilities approach is important and worth giving a good look in any new language. It is a feature you cannot very well add late.
-Lex
Lex Spoon wrote:
Michael van der Gulik squeakml@gulik.co.nz writes:
Does anybody have code (particularly VM modifications) which allow Capabilities in Squeak?
In particular, I'm referring to code that implements stuff described on this page: http://minnow.cc.gatech.edu/squeak/uploads/2074/sandbox.html
This is my old "Islands" project, [...] Full information about Islands is available at this page:
http://minnow.cc.gatech.edu/squeak/2074[...] I still think the basic approach is good. The next thing I would do, were I to continue, would be to get rid of the dynamically bound global variables, and instead to have separate, static namespaces.
That does sound good.
The reason for the current approach--i.e. all global references are bound indirectly through the currently active island--is that compiled code can be reused across multiple islands. In retrospect, it would be better to maintain conceptual pruity and simply recompile any reused code.
A different compilation strategy would still allow compiled code to be shared -- by treating these the way other languages treat captured outer lexical variables.
More broadly, I still think the object capabilities approach is important and worth giving a good look in any new language. It is a feature you cannot very well add late.
It has indeed been hard to add objcaps to Squeak after the fact, or rather to subtract out the non-objcap parts of the language. (Motto: "Don't add security, remove insecurity.") Other efforts have yielded varying results. Securing Java to create Joe-E[1] looks quite good, and we have recently been using this successfully within HP. Although Java is much more "object-oriented" than Scheme or OCaml, W7[2] and Emily[3] were much easier than Joe-E, whereas securing Common Lisp[4] was hard enough that the effort seems to have been abandoned. The effort to secure Mozart/Oz is proceeding slowly, but has yielded one of the best documents[5] about the issues in retrofitting objcaps into an existing language. I am also hopeful about a new effort to secure Python[6].
All these efforts have freshly encountered many of the same issues. It would be good if they could learn more from each other. A secure Squeak-like language would still be awesome. Perhaps we should have a workshop about retrofitting objcaps into existing languages?
[1] http://www.joe-e.org/ [2] http://mumble.net/~jar/pubs/secureos/ [3] http://www.hpl.hp.com/techreports/2006/HPL-2006-116.html [4] http://www.eros-os.org/pipermail/e-lang/2005-August/010923.html [5] http://www.info.ucl.ac.be/~pvr/oze.pdf [6] http://sayspy.blogspot.com/2006/07/security-design-doc-using-object.html
Oh yeah, and CaPerl[7,8], which looks more plausible than I could ever have guessed.
[7] http://caperl.links.org/ [8] https://www.opengroup.org/projects/jericho/uploads/40/8299/8_-_JFChallenge_L...
Mark S. Miller wrote:
All these efforts have freshly encountered many of the same issues. It would be good if they could learn more from each other. A secure Squeak-like language would still be awesome. Perhaps we should have a workshop about retrofitting objcaps into existing languages?
[1] http://www.joe-e.org/ [2] http://mumble.net/~jar/pubs/secureos/ [3] http://www.hpl.hp.com/techreports/2006/HPL-2006-116.html [4] http://www.eros-os.org/pipermail/e-lang/2005-August/010923.html [5] http://www.info.ucl.ac.be/~pvr/oze.pdf [6] http://sayspy.blogspot.com/2006/07/security-design-doc-using-object.html
Wow! I have reading to do.
Thanks for the references!
Michael.
The reason for the current approach--i.e. all global references are bound indirectly through the currently active island--is that compiled code can be reused across multiple islands. In retrospect, it would be better to maintain conceptual pruity and simply recompile any reused code.
A different compilation strategy would still allow compiled code to be shared -- by treating these the way other languages treat captured outer lexical variables.
Yes, though for what I think you are suggesting, you would want to have outer variables that are not global -- i.e., nested classes.
So that gives three solutions to the problem of having multiple (for example) "World" variables. The original Islands turns the global variable World into an island-scoped variable, with a dynamic lookup indirected through the current island. This has the advantage that existing code needs little modification; however, it has the significant cost that you have to be very careful about which island is currently installed as the active one. That turns out to be rather fiddly and error prone.
The nice thing, though, before moving on, is that a "global" variable reference gives no new authority! Thus, if you are reviewing the code of a class, you do not even need to look at normal global references. Instead, you only need to look at "hard-bound" references, i.e. statically bound references, which are rare.
Anyway, the middle-level solution I propose above is to have multiple global namespaces, and to recompile classes for each namespace. This way you can still reuse classes that refer directly to World, but you have to fiddle with recompiling them. I would try this first, just because it is so unobtrusive to the existing language and libraries.
Perhaps we should go all the way, though, and explore nested classes. The only issue there is that it is a major change. Implementing it in the language is not a big deal, but updating all the browsers and debuggers and so on looks like a lot of work, especially if you try to achieve anything like Smalltalk's level of tool quality. Still, maybe you have to go this far if you want to take the hard line on lexical scope and get a usable, security-sensible system.
More broadly, I still think the object capabilities approach is important and worth giving a good look in any new language. It is a feature you cannot very well add late.
It has indeed been hard to add objcaps to Squeak after the fact, or rather to subtract out the non-objcap parts of the language. (Motto: "Don't add security, remove insecurity.") Other efforts have yielded varying results.
Thanks for the excellent overview. I was only aware of some of these.
-Lex
On 18 Oct 2006 19:36:02 -0400, Lex Spoon lex@cc.gatech.edu wrote:
Perhaps we should go all the way, though, and explore nested classes. The only issue there is that it is a major change. Implementing it in the language is not a big deal, but updating all the browsers and debuggers and so on looks like a lot of work, especially if you try to achieve anything like Smalltalk's level of tool quality. Still, maybe you have to go this far if you want to take the hard line on lexical scope and get a usable, security-sensible system.
Nested classes? What do you mean? Are you talking about nested Namespaces ( org.squeak.kernel.numbers.SmallInteger)?
Michael.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Michael
Michael van der Gulik wrote:
On 18 Oct 2006 19:36:02 -0400, *Lex Spoon* <lex@cc.gatech.edu mailto:lex@cc.gatech.edu> wrote:
Perhaps we should go all the way, though, and explore nested classes. The only issue there is that it is a major change. Implementing it in the language is not a big deal, but updating all the browsers and debuggers and so on looks like a lot of work, especially if you try to achieve anything like Smalltalk's level of tool quality. Still, maybe you have to go this far if you want to take the hard line on lexical scope and get a usable, security-sensible system.Nested classes? What do you mean? Are you talking about nested Namespaces (org.squeak.kernel.numbers.SmallInteger)?
Have you read papers about E? Have you read Mark Miller's dissertation? http://www.erights.org/talks/thesis/index.html
"Nested classes", "nested functions" or lexical scoping is related to the unusual (like in lambda-calculus, like in pi-calculus, etc) rules which determine which variable names are visible in the particular point in the source code. It may seem "weird" but it is one mechanism which can be used to build secure language (in the sense of E).
Can we implement Point in Smalltalk and E? Yes.
What about ConstantPoint? Constant point is such an object which behaves exactly like a Point except for that it cannot be moved (once it is created). Can we implement such a thing in Smalltalk? Yes, it is "enough" to implement E interpreter in Squeak and then you are done. Can we implement constant point directly in E? Yes, it is trivial (few lines of code).
Or am I wrong? Is it somehow possible to implement ConstantPoint in Smalltalk? - -- Matej Ko?ík icq: 300133844, skype: matej_kosik, sarkan@jabber.sk
Or am I wrong? Is it somehow possible to implement ConstantPoint in Smalltalk?
If you need a point that do not move, do not send #move messages to the point. Nothing more efficient than the action that has not been done. (if you have points that move and points that do not move... it is very provable to have a "failure" or a wrong point in the wrong place)
best, Ale.
----- Original Message ----- From: "Matej Kosik" kosik@fiit.stuba.sk To: "The general-purpose Squeak developers list" squeak-dev@lists.squeakfoundation.org Sent: Thursday, October 19, 2006 4:28 AM Subject: Re: Retrofitting objcaps
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Michael
Michael van der Gulik wrote:
On 18 Oct 2006 19:36:02 -0400, *Lex Spoon* <lex@cc.gatech.edu mailto:lex@cc.gatech.edu> wrote:
Perhaps we should go all the way, though, and explore nested
classes.
The only issue there is that it is a major change. Implementing it
in
the language is not a big deal, but updating all the browsers and debuggers and so on looks like a lot of work, especially if you try
to
achieve anything like Smalltalk's level of tool quality. Still,
maybe
you have to go this far if you want to take the hard line on lexical scope and get a usable, security-sensible system.Nested classes? What do you mean? Are you talking about nested Namespaces (org.squeak.kernel.numbers.SmallInteger)?
Have you read papers about E? Have you read Mark Miller's dissertation? http://www.erights.org/talks/thesis/index.html
"Nested classes", "nested functions" or lexical scoping is related to the unusual (like in lambda-calculus, like in pi-calculus, etc) rules which determine which variable names are visible in the particular point in the source code. It may seem "weird" but it is one mechanism which can be used to build secure language (in the sense of E).
Can we implement Point in Smalltalk and E? Yes.
What about ConstantPoint? Constant point is such an object which behaves exactly like a Point except for that it cannot be moved (once it is created). Can we implement such a thing in Smalltalk? Yes, it is "enough" to implement E interpreter in Squeak and then you are done. Can we implement constant point directly in E? Yes, it is trivial (few lines of code).
Or am I wrong? Is it somehow possible to implement ConstantPoint in Smalltalk?
Matej Ko?ík icq: 300133844, skype: matej_kosik, sarkan@jabber.sk -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFNykgL+CaXfJI/hgRAs3NAKDDbxKxZpJ+3TUGrgDrxNMZkHiDAACggP4s l5V8WAp1Yf8Wzy6FOuxSfm4= =CQjN -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Alejandro
Alejandro F. Reimondo wrote:
Or am I wrong? Is it somehow possible to implement ConstantPoint in Smalltalk?
If you need a point that do not move, do not send #move messages to the point.
If I were the only one who would like to use that object than it would be partially acceptable. But as soon as I would give others the permission to that constant-point, I would like to *enforce* my particular policy I had in mind.
If I do not want others to move that point then I do not want to give them authority to move that point.
If I do not want other system to write more than 100 character log to some file, then I do not want to give it authority to write more than 100 characters.
(other examples could be invented)
In Smalltalk such things are not enforcable. In E they are. Why is the ability to enforce some policy important? It is usually useful when one wants to cooperate with parties but he/she/it does not fully trust them. In day to day reality this happens all the time.
Nothing more efficient than the action that has not been done. (if you have points that move and points that do not move... it is very provable to have a "failure" or a wrong point in the wrong place)
- -- Matej Kosik icq: 300133844, skype: matej_kosik, sarkan@jabber.sk
Matej Kosik wrote:
What about ConstantPoint? Constant point is such an object which behaves exactly like a Point except for that it cannot be moved (once it is created). Can we implement such a thing in Smalltalk? Yes, it is "enough" to implement E interpreter in Squeak and then you are done. Can we implement constant point directly in E? Yes, it is trivial (few lines of code).
Or am I wrong? Is it somehow possible to implement ConstantPoint in Smalltalk?
Sure. It's not even hard and I can think of at least three methods depending on how far you want to go with the level of support. The easiest (and most illustrative, though not directly comparable) method is this:
makePoint: aPoint "Create a constant point object" | pointClass pointObj | pointClass := aPoint newUniclass. pointClass compile: 'x ^', aPoint x printString. pointClass compile: 'y ^', aPoint y printString. ^pointClass new
The other two methods are: * Use literal sharing, e.g., the identical x, and y values. The difference to the above is that both x and y would be parsed again by the compiler and therefore may not be identical to the input * Use blocks as accessors which would also use lexical scoping rules and be the most truthful (but least readable and hardest to implement directly) solution.
The main thing that's missing in Squeak to support this more easily is some "literal object" syntax by which you could create an anonymous class in a method and instantiate that class directly. This would allow you to reuse the same class over and over again, but it would not be much different from the above.
Cheers, - Andreas
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Andreas,
Andreas Raab wrote:
Matej Kosik wrote:
What about ConstantPoint? Constant point is such an object which behaves exactly like a Point except for that it cannot be moved (once it is created). Can we implement such a thing in Smalltalk? Yes, it is "enough" to implement E interpreter in Squeak and then you are done. Can we implement constant point directly in E? Yes, it is trivial (few lines of code).
Or am I wrong? Is it somehow possible to implement ConstantPoint in Smalltalk?
Sure. It's not even hard and I can think of at least three methods depending on how far you want to go with the level of support. The easiest (and most illustrative, though not directly comparable) method is this:
makePoint: aPoint "Create a constant point object" | pointClass pointObj | pointClass := aPoint newUniclass. pointClass compile: 'x ^', aPoint x printString. pointClass compile: 'y ^', aPoint y printString. ^pointClass new
This is interesting. Thank you. Although I do not have `newUniclass' method directly in my image, there is something similar. So if I do this:
pointClass := Object newUniqueClassInstVars: '' classInstVars: ''. pointClass compile: 'x ^ 100'. pointClass compile: 'y ^ 100'. point := pointClass new.
I can indeed see that
point x
gives
100
and point y
gives
100
However, if I passed reference to the `point' object to you (to your code), were I able to stop you (your code) from doing
point class compile: 'x ^ whatever'
? If not, then this is not the solution to the original problem. It would be necessary to ban the "compile:" class method. And since Smalltalk has many crooked lanes, we could again miss some other vulnerabilities.
Does `point' have a class? Yes. Its name is
point class
In my case `Object1'. What would prevent anyone from doing this
(Smalltalk at: #Object1) compile: 'x ^ 10'
this should again be banned. It must be enforced that something like that is impossible. I am afraid that the number of possible vulnerabilities is overwhelming. Although this is related to the "libraries" rather than to the language, the question is, whether the "libraries" can be "tamed" or whether the whole thing should be thrown away and everything restarted with security in mind from the begining. In E they've done it. I believe. The tendency to keep current languages and "adding security" is caused by the fact that we love Smalltalk (Python, Ruby, Java, Erlang, ...) and do not want to turn back to it. I am not proposing everyone to swich to E, I am not part of the E project, I am only amazed, the question is, if we are really concerned with security, whether such switch (directly or indirectly) would not be necessary. Adding convenience/nice gui/great gui tools to E could be possible whereas adding security to Smalltalk/Java/Python/Ruby/Erlang may turn out to be - - either impossible - - or we will end up with essentially some almost E-quivalent (and thus contributing to world fragmentation)
The other two methods are:
- Use literal sharing, e.g., the identical x, and y values. The
difference to the above is that both x and y would be parsed again by the compiler and therefore may not be identical to the input
- Use blocks as accessors which would also use lexical scoping rules and
be the most truthful (but least readable and hardest to implement directly) solution.
The main thing that's missing in Squeak to support this more easily is some "literal object" syntax by which you could create an anonymous class in a method and instantiate that class directly. This would allow you to reuse the same class over and over again, but it would not be much different from the above.
Cheers,
- Andreas
- -- Matej Kosik icq: 300133844, skype: matej_kosik, sarkan@jabber.sk
Can anyone help me to get this to work.
MczInstaller installStream: (ReadStream on: (('http://bugs.impara.de/file_download.php?file_id=2176&type=bug') asUrl retrieveContents content))
I have tried a wide variety of variants
many thanks in advance
Keith
___________________________________________________________ Try the all-new Yahoo! Mail. "The New Version is radically easier to use" The Wall Street Journal http://uk.docs.yahoo.com/nowyoucan.html
Keith Hodges puso en su mail :
Can anyone help me to get this to work.
MczInstaller installStream: (ReadStream on: (('http://bugs.impara.de/file_download.php?file_id=2176&type=bug') asUrl retrieveContents content))
I have tried a wide variety of variants
many thanks in advance
Keith
Clicking in the link automatic download the Join-kph.1.mcz (on Mac)
From here, you could drag and drop or fileIn.
By the way , what do the .mcz ?
__________________________________________________ Pregunt�. Respond�. Descubr�. Todo lo que quer�as saber, y lo que ni imaginabas, est� en Yahoo! Respuestas (Beta). �Probalo ya! http://www.yahoo.com.ar/respuestas
Ok I did it after 4 hours of trying
MczInstaller installStream: (('http://bugs.impara.de/file_download.php?file_id=2176&type=bug') asUrl retrieveContents contentStream)
many thanks in advance
Keith
___________________________________________________________ Copy addresses and emails from any email account to Yahoo! Mail - quick, easy and free. http://uk.docs.yahoo.com/trueswitch2.html
Matej Kosik wrote:
However, if I passed reference to the `point' object to you (to your code), were I able to stop you (your code) from doing
point class compile: 'x ^ whatever'
? If not, then this is not the solution to the original problem. It would be necessary to ban the "compile:" class method. And since Smalltalk has many crooked lanes, we could again miss some other vulnerabilities.
Of course you could prevent hat but the art is in restructuring the class libraries to serve this goal. The trivial solution is to override over access to the method dictionary by, e.g.,
ConstanstPoint class>>addSelector: selector withMethod: method self error: 'Poop'.
or somesuch. That's not enough by far but it shows the general direction (Lex has a lot of that in his whitepaper).
Does `point' have a class? Yes. Its name is
point class
In my case `Object1'. What would prevent anyone from doing this
(Smalltalk at: #Object1) compile: 'x ^ 10'
this should again be banned.
This is why I used #newUniclass. In Tweak, this creates an anonymous subclass which is marked by an asterisk in front so
Point newUniclass name -> #'*Point'
and it's not in the global system dictionary either. Of course there are currently many other ways of getting your hands at it, like enumerating the subclasses of point, or just all objects in memory. All of these need to be tamed.
It must be enforced that something like that is impossible. I am afraid that the number of possible vulnerabilities is overwhelming. Although this is related to the "libraries" rather than to the language, the question is, whether the "libraries" can be "tamed" or whether the whole thing should be thrown away and everything restarted with security in mind from the begining. In E they've done it. I believe. The tendency to keep current languages and "adding security" is caused by the fact that we love Smalltalk (Python, Ruby, Java, Erlang, ...) and do not want to turn back to it. I am not proposing everyone to swich to E, I am not part of the E project, I am only amazed, the question is, if we are really concerned with security, whether such switch (directly or indirectly) would not be necessary. Adding convenience/nice gui/great gui tools to E could be possible whereas adding security to Smalltalk/Java/Python/Ruby/Erlang may turn out to be
- either impossible
- or we will end up with essentially some almost E-quivalent (and thus
contributing to world fragmentation)
The easier thing to do is to introduce an arbitrary layer on top of our current system and use that as a baseline for programming. As you progress, you move more and more from the layers below (which effectively form your TCB) to the layers on top.
Cheers, - Andreas
On 10/19/06, Matej Kosik kosik@fiit.stuba.sk wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Michael
Michael van der Gulik wrote:
On 18 Oct 2006 19:36:02 -0400, *Lex Spoon* <lex@cc.gatech.edu mailto:lex@cc.gatech.edu> wrote:
Perhaps we should go all the way, though, and explore nestedclasses.
The only issue there is that it is a major change. Implementing itin
the language is not a big deal, but updating all the browsers and debuggers and so on looks like a lot of work, especially if you tryto
achieve anything like Smalltalk's level of tool quality. Still,maybe
you have to go this far if you want to take the hard line on lexical scope and get a usable, security-sensible system.Nested classes? What do you mean? Are you talking about nested Namespaces (org.squeak.kernel.numbers.SmallInteger)?
Have you read papers about E? Have you read Mark Miller's dissertation? http://www.erights.org/talks/thesis/index.html
His dissertation does not contain the phrase "nested class". I've just searched through it.
Michael.
Lex Spoon wrote:
Perhaps we should go all the way, though, and explore nested classes.
Michael van der Gulik wrote:
Nested classes? What do you mean? Are you talking about nested Namespaces (org.squeak.kernel.numbers.SmallInteger)?
Matej Kosik wrote:
Have you read papers about E? Have you read Mark Miller's dissertation? http://www.erights.org/talks/thesis/index.html http://www.erights.org/talks/thesis/index.html
Michael van der Gulik wrote:
His dissertation does not contain the phrase "nested class". I've just searched through it.
From section 6.4:
Any imperative language with lexically nested object definitions, including Smalltalk's blocks, T [RA82], Beta [Mad00], Java's inner classes, and Emerald [HRB+87] covered briefly in Related Work Section 24.6, supports the easy definition of multiple facets sharing state.
In any case, the point in not inner/nested classes per se. The point is lexically nested object definitions.
Lex Spoon wrote:
Michael van der Gulik squeakml@gulik.co.nz writes:
Hi all.
Does anybody have code (particularly VM modifications) which allow Capabilities in Squeak?
In particular, I'm referring to code that implements stuff described on this page: http://minnow.cc.gatech.edu/squeak/uploads/2074/sandbox.html
This is my old "Islands" project, done during an summer internship with the Squeak group in 2000. The group was just starting to work on sharing EToy's. For example, that was when the SuperSwiki and the web-browser plugin were developed.
Islands is used in Croquet. I'll have to see how they're used there and what changes have been made.
Do you mind if I grab your code and run with it? Also, what license is this released under? MIT?
Many thanks! Michael.
squeak-dev@lists.squeakfoundation.org
-
Alejandro F. Reimondo -
Andreas Raab -
Bert Freudenberg -
David T. Lewis -
Edgar J. De Cleene -
Josh Gargus -
Keith Hodges -
Klaus D. Witzel -
Lex Spoon -
Mark S. Miller -
Matej Kosik -
Michael van der Gulik -
Michael van der Gulik -
Robert Withers