Hi all,
I've just uploaded a package to the Inbox for review by the community:
http://source.squeak.org/inbox/Mirrors-cwp.2.mcz
It's a simple first step at creating a mirror API for Squeak. I've only implemented ObjectMirror and ObjectVmMirror, which provide high- and low-level reflection on an object. This implementation has a twist, however: it sends no messages to the object it's reflecting. That allows us to reflect on network proxies, ORM stubs, mock objects and the like without triggering any state changes.
As a proof of the concept, I've implemented a rudimentary non-invasive MirrorInspector. The main thing that's missing is non-invasive printing, which would take a bit of work to implement, but isn't necessary for a demo. To see it in action do try the following in a workspace.
WARNING: Read the comments carefully, and don't do this in an image you care about!
"This creates a very dangerous object" Object subclass: #Nuke instanceVariableNames: 'one two three' classVariableNames: '' poolDictionaries: '' category: 'Test'. class := Smalltalk at: #Nuke. class superclass: nil. inst := class basicNew.
"This is harmless" MirrorInspector inspect: inst.
"This will crash your image" inst yourself.
The implementation uses primitive 188 to directly execute CompiledMethods with the reflected object as the receiver, thus giving us access to the object's state without sending it a message.
So, what do you think?
Colin
On 22 May 2012 08:07, Colin Putney colin@wiresong.com wrote:
Hi all,
I've just uploaded a package to the Inbox for review by the community:
http://source.squeak.org/inbox/Mirrors-cwp.2.mcz
It's a simple first step at creating a mirror API for Squeak. I've only implemented ObjectMirror and ObjectVmMirror, which provide high- and low-level reflection on an object. This implementation has a twist, however: it sends no messages to the object it's reflecting. That allows us to reflect on network proxies, ORM stubs, mock objects and the like without triggering any state changes.
As a proof of the concept, I've implemented a rudimentary non-invasive MirrorInspector. The main thing that's missing is non-invasive printing, which would take a bit of work to implement, but isn't necessary for a demo. To see it in action do try the following in a workspace.
WARNING: Read the comments carefully, and don't do this in an image you care about!
"This creates a very dangerous object" Object subclass: #Nuke instanceVariableNames: 'one two three' classVariableNames: '' poolDictionaries: '' category: 'Test'. class := Smalltalk at: #Nuke. class superclass: nil. inst := class basicNew.
"This is harmless" MirrorInspector inspect: inst.
"This will crash your image" inst yourself.
The implementation uses primitive 188 to directly execute CompiledMethods with the reflected object as the receiver, thus giving us access to the object's state without sending it a message.
So, what do you think?
MessageShunt can't load because it has a nil superclass, causing an MNU for #allowsSubInstVars in ClassBuilder >> #validateInstVars:from:forSuper: because the last arg - newSuper - is nil.
frank
Colin
On 2012-05-22, at 3:09 AM, Frank Shearar wrote:
MessageShunt can't load because it has a nil superclass, causing an MNU for #allowsSubInstVars in ClassBuilder >> #validateInstVars:from:forSuper: because the last arg - newSuper - is nil.
Drat. That turns out to more of an issue than I thought. I've uploaded a new version that has ProtoObject as a superclass.
http://source.squeak.org/inbox/Mirrors-cwp.3.mcz
Colin
On 22 May 2012 16:06, Colin Putney colin@wiresong.com wrote:
On 2012-05-22, at 3:09 AM, Frank Shearar wrote:
MessageShunt can't load because it has a nil superclass, causing an MNU for #allowsSubInstVars in ClassBuilder >> #validateInstVars:from:forSuper: because the last arg - newSuper - is nil.
Drat. That turns out to more of an issue than I thought. I've uploaded a new version that has ProtoObject as a superclass.
Ah yes, that works, and the tests pass. Now to actually read the code :) Thanks!
frank
Colin
On Tue, May 22, 2012 at 12:07 AM, Colin Putney colin@wiresong.com wrote:
Hi all,
I've just uploaded a package to the Inbox for review by the community:
http://source.squeak.org/inbox/Mirrors-cwp.2.mcz
It's a simple first step at creating a mirror API for Squeak. I've only implemented ObjectMirror and ObjectVmMirror, which provide high- and low-level reflection on an object. This implementation has a twist, however: it sends no messages to the object it's reflecting. That allows us to reflect on network proxies, ORM stubs, mock objects and the like without triggering any state changes.
As a proof of the concept, I've implemented a rudimentary non-invasive MirrorInspector. The main thing that's missing is non-invasive printing, which would take a bit of work to implement, but isn't necessary for a demo. To see it in action do try the following in a workspace.
WARNING: Read the comments carefully, and don't do this in an image you care about!
"This creates a very dangerous object" Object subclass: #Nuke instanceVariableNames: 'one two three' classVariableNames: '' poolDictionaries: '' category: 'Test'. class := Smalltalk at: #Nuke. class superclass: nil. inst := class basicNew.
"This is harmless" MirrorInspector inspect: inst.
"This will crash your image" inst yourself.
The implementation uses primitive 188 to directly execute CompiledMethods with the reflected object as the receiver, thus giving us access to the object's state without sending it a message.
In part you can use the light-weigth mirror methods in ContextPart, these take the object operated on as an argument. Non-invasive printing could be done via execution simulation. I've already modified the Squeak debugger in Qwaq/Teleplace images to use these light-weight mirror primitives so that messages are not sent to receivers when simulating execution in the debugger (i.e. when doing send instead of step). If the mirror primitives now work on the interpreter I can fold this into trunk. David, do the mirror primitive tests pass on the interpreter?
Doing this is also a good idea in general since it allows the debugger to correctly debug proxies also.
So, what do you think?
Colin
On Tue, May 22, 2012 at 3:24 PM, Eliot Miranda eliot.miranda@gmail.com wrote:
In part you can use the light-weigth mirror methods in ContextPart, these take the object operated on as an argument. Non-invasive printing could be done via execution simulation. I've already modified the Squeak debugger in Qwaq/Teleplace images to use these light-weight mirror primitives so that messages are not sent to receivers when simulating execution in the debugger (i.e. when doing send instead of step). If the mirror primitives now work on the interpreter I can fold this into trunk. David, do the mirror primitive tests pass on the interpreter?
Yeah, I looked at those, but they seem unnecessary given the existence of primitive 188, (i.e., CompiledMethod class>>receiver:withArguments:executeMethod:). With that one primitive we can do anything we need, with all the logic in the image, instead of the VM.
Is there some reason you prefer separate primitives for all these operations? You called them light-weight—what do you mean by that?
Doing this is also a good idea in general since it allows the debugger to correctly debug proxies also.
Yes!
I guess we have two separate issues here, which I conflated in my first post. On the one hand, there's non-invasive tool support. That would be good, and it can be implemented in various ways. Your debugger changes sound like a good first step in that direction. We could also implement an inspector based on the light-weight mirror primitives.
On the other hand, there's the question of a mirror API. That's one of the things I really like about the Self-Strongtalk-Newspeak-Dart thread of language design, and hey, it would be great to have that in Squeak as well. (We stole Morphic from Self, why not mirrors?)
If we did decided to move to a mirror-based API, then non-invasive mirrors seems like the way to go, and this approach based on prim 188 seems like a good way to implement that. This experiment makes me think it's feasible. Laying aside the (very real) issues of how to get from here to there, is this a direction worth exploring, or is Smalltalk-80-style reflection good enough?
Colin
On 23 May 2012 03:59, Colin Putney colin@wiresong.com wrote:
On Tue, May 22, 2012 at 3:24 PM, Eliot Miranda eliot.miranda@gmail.com wrote:
In part you can use the light-weigth mirror methods in ContextPart, these take the object operated on as an argument. Non-invasive printing could be done via execution simulation. I've already modified the Squeak debugger in Qwaq/Teleplace images to use these light-weight mirror primitives so that messages are not sent to receivers when simulating execution in the debugger (i.e. when doing send instead of step). If the mirror primitives now work on the interpreter I can fold this into trunk. David, do the mirror primitive tests pass on the interpreter?
Yeah, I looked at those, but they seem unnecessary given the existence of primitive 188, (i.e., CompiledMethod class>>receiver:withArguments:executeMethod:). With that one primitive we can do anything we need, with all the logic in the image, instead of the VM.
Is there some reason you prefer separate primitives for all these operations? You called them light-weight—what do you mean by that?
Doing this is also a good idea in general since it allows the debugger to correctly debug proxies also.
Yes!
I guess we have two separate issues here, which I conflated in my first post. On the one hand, there's non-invasive tool support. That would be good, and it can be implemented in various ways. Your debugger changes sound like a good first step in that direction. We could also implement an inspector based on the light-weight mirror primitives.
On the other hand, there's the question of a mirror API. That's one of the things I really like about the Self-Strongtalk-Newspeak-Dart thread of language design, and hey, it would be great to have that in Squeak as well. (We stole Morphic from Self, why not mirrors?)
If we did decided to move to a mirror-based API, then non-invasive mirrors seems like the way to go, and this approach based on prim 188 seems like a good way to implement that. This experiment makes me think it's feasible. Laying aside the (very real) issues of how to get from here to there, is this a direction worth exploring, or is Smalltalk-80-style reflection good enough?
Colin
Nice trick with 188 prim.. yes, actually using this prim you can attach any behavior to any object, without it's permission.. so why not mirrors ;)
I sure this looks not very good in the eyes of pro-security guys, but as to me it is fine. Maybe mirror primitives, mentioned by Eliot is kind of providing the same but in more conservative/secure way and probably a bit faster. But unless we ban primitive 188, i don't see why we need something more/else for mirrors.
And yes, i vote for mirrors.
On Tue, May 22, 2012 at 6:59 PM, Colin Putney colin@wiresong.com wrote:
On Tue, May 22, 2012 at 3:24 PM, Eliot Miranda eliot.miranda@gmail.com wrote:
In part you can use the light-weigth mirror methods in ContextPart, these take the object operated on as an argument. Non-invasive printing could
be
done via execution simulation. I've already modified the Squeak
debugger in
Qwaq/Teleplace images to use these light-weight mirror primitives so that messages are not sent to receivers when simulating execution in the
debugger
(i.e. when doing send instead of step). If the mirror primitives now
work
on the interpreter I can fold this into trunk. David, do the mirror primitive tests pass on the interpreter?
Yeah, I looked at those, but they seem unnecessary given the existence of primitive 188, (i.e., CompiledMethod class>>receiver:withArguments:executeMethod:). With that one primitive we can do anything we need, with all the logic in the image, instead of the VM.
Is there some reason you prefer separate primitives for all these operations? You called them light-weight—what do you mean by that?
Doing this is also a good idea in general since it allows the debugger
to correctly debug proxies also.
Yes!
I guess we have two separate issues here, which I conflated in my first post. On the one hand, there's non-invasive tool support. That would be good, and it can be implemented in various ways. Your debugger changes sound like a good first step in that direction. We could also implement an inspector based on the light-weight mirror primitives.
On the other hand, there's the question of a mirror API. That's one of the things I really like about the Self-Strongtalk-Newspeak-Dart thread of language design, and hey, it would be great to have that in Squeak as well. (We stole Morphic from Self, why not mirrors?)
If we did decided to move to a mirror-based API, then non-invasive mirrors seems like the way to go, and this approach based on prim 188 seems like a good way to implement that. This experiment makes me think it's feasible. Laying aside the (very real) issues of how to get from here to there, is this a direction worth exploring, or is Smalltalk-80-style reflection good enough?
To not address your core questions, use of 188 is I think clunky. Instead, implementing the mirror primitives in the mirrors seems simple and efficient. Putting the mirror primitives in ContextPart is insecure, since anyone can to thisContext object: foo instVarAt: n. But fixing this means a major design overhaul of the system re debugging and mirrors, so it can stand for now. (After all anyone can do instVarAt: now, so we're a ways away from a secure Newspeak-style sandbox).
Colin
On Wed, May 23, 2012 at 11:54 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
To not address your core questions, use of 188 is I think clunky. Instead, implementing the mirror primitives in the mirrors seems simple and efficient.
You find it clunky because you can't just call the primitive, you have to do a manual lookup to find the appropriate CompiledMethod first? Fair enough. I don't think it's an issue with mirrors, because the mirror encapsulates that clunky lookup behaviour, and lets us make us of it through a clean and simple protocol. I prefer a smaller, more general VM/image interface, with as much of the logic as possible in the image.
But 188 does have the following practical advantages:
- it's present in older VMs that are already out there - in the future we'll be able to make changes at the image level, rather than having to ship new VMs
The other thread on your debugger improvements would be moot if they had been based on 188 instead of custom primitives.
Putting the mirror primitives in ContextPart is insecure, since anyone can to thisContext object: foo instVarAt: n. But fixing this means a major design overhaul of the system re debugging and mirrors, so it can stand for now. (After all anyone can do instVarAt: now, so we're a ways away from a secure Newspeak-style sandbox).
Agreed.
Colin
On 24 May 2012 00:29, Colin Putney colin@wiresong.com wrote:
On Wed, May 23, 2012 at 11:54 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
To not address your core questions, use of 188 is I think clunky. Instead, implementing the mirror primitives in the mirrors seems simple and efficient.
You find it clunky because you can't just call the primitive, you have to do a manual lookup to find the appropriate CompiledMethod first?
Sorry, but why you need a lookup? Do i miss something? I didn't checked your implementation.
For instance, Pharo no longer uses shortcut for #class message, sending a class to an object is a normal message send with usual lookup procedure. Therefore , sending #class to an object does not guarantees that it will answer its class (especially, if it can be simply not implemented).
Now i don't see where you need to do a manual lookup in following:
(Mirror on: arbitraryObject) subjectClass
Mirror>>subjectClass ^ CompiledMethod receiver: subject withArguments: argArray executeMethod: answerSubjectClass
(here the subject is an object which mirror reflects).
Fair enough. I don't think it's an issue with mirrors, because the mirror encapsulates that clunky lookup behaviour, and lets us make us of it through a clean and simple protocol. I prefer a smaller, more general VM/image interface, with as much of the logic as possible in the image.
But 188 does have the following practical advantages:
- it's present in older VMs that are already out there
- in the future we'll be able to make changes at the image level,
rather than having to ship new VMs
The other thread on your debugger improvements would be moot if they had been based on 188 instead of custom primitives.
Indeed, i think 188 having a great potential if used by good hands.
Putting the mirror primitives in ContextPart is insecure, since anyone can to thisContext object: foo instVarAt: n. But fixing this means a major design overhaul of the system re debugging and mirrors, so it can stand for now. (After all anyone can do instVarAt: now, so we're a ways away from a secure Newspeak-style sandbox).
Agreed.
Colin
ok, i checked your implementation and now i see where you doing lookup
receiver: anObject perform: aSelector arguments: anArray | cm | cm := self compiledMethodAt: aSelector ifAbsent: [^ anObject doesNotUnderstand: (Message selector: aSelector arguments: anArray)]. ^ self receiver: anObject withArguments: anArray executeMethod: cm
But you can avoid it, consider a following:
ObjectVmMirror>>fixedAt: anInteger ^ Reflection receiver: object perform: #fixedAt: arguments: {anInteger}
you can rewrite it as:
ObjectVmMirror>>fixedAt: anInteger ^ self execute: GetFixedSlotMethod arguments: {anInteger}
where GetFixedSlotMethod is a class variable holding an appropriate CompiledMethod instance, which you can initialize just once at class initialization (and reinitialize if it changes).
and #execute:arguments: is method which using prim 188 for executing given method with object as receiver.
On Wed, May 23, 2012 at 5:13 PM, Igor Stasenko siguctua@gmail.com wrote:
ok, i checked your implementation and now i see where you doing lookup
[snip]
ObjectVmMirror>>fixedAt: anInteger ^ self execute: GetFixedSlotMethod arguments: {anInteger}
where GetFixedSlotMethod is a class variable holding an appropriate CompiledMethod instance, which you can initialize just once at class initialization (and reinitialize if it changes).
and #execute:arguments: is method which using prim 188 for executing given method with object as receiver.
Sure, that would be cleaner and a bit faster. I'll make that change, thanks.
I don't think it'll please Eliot though - we still have to get the CompiledMethod from *somewhere* even if it's a class variable.
Colin
On 24 May 2012 02:34, Colin Putney colin@wiresong.com wrote:
On Wed, May 23, 2012 at 5:13 PM, Igor Stasenko siguctua@gmail.com wrote:
ok, i checked your implementation and now i see where you doing lookup
[snip]
ObjectVmMirror>>fixedAt: anInteger ^ self execute: GetFixedSlotMethod arguments: {anInteger}
where GetFixedSlotMethod is a class variable holding an appropriate CompiledMethod instance, which you can initialize just once at class initialization (and reinitialize if it changes).
and #execute:arguments: is method which using prim 188 for executing given method with object as receiver.
Sure, that would be cleaner and a bit faster. I'll make that change, thanks.
I don't think it'll please Eliot though - we still have to get the CompiledMethod from *somewhere* even if it's a class variable.
i like the concept. it is quite elegant, to my look: - things like GetFixedSlotMethod & others is actually a VM contracts which language side must know about to properly reflect the objects without sending any message to them (same as VM does).
So you end up with things like
GetObjectBehaviorMethod GetObjectVarSizeMethod
GetBehaviorSuperclassMethod GetBehaviorMDMethod GetBehaviorFormatMethod etc
i would even create a separate class for that.. which will hold all those contracts. So it can be useful for educational purposes too.
Colin
On Tue, May 22, 2012 at 03:24:34PM -0700, Eliot Miranda wrote:
On Tue, May 22, 2012 at 12:07 AM, Colin Putney colin@wiresong.com wrote:
Hi all,
I've just uploaded a package to the Inbox for review by the community:
http://source.squeak.org/inbox/Mirrors-cwp.2.mcz
It's a simple first step at creating a mirror API for Squeak. I've only implemented ObjectMirror and ObjectVmMirror, which provide high- and low-level reflection on an object. This implementation has a twist, however: it sends no messages to the object it's reflecting. That allows us to reflect on network proxies, ORM stubs, mock objects and the like without triggering any state changes.
As a proof of the concept, I've implemented a rudimentary non-invasive MirrorInspector. The main thing that's missing is non-invasive printing, which would take a bit of work to implement, but isn't necessary for a demo. To see it in action do try the following in a workspace.
WARNING: Read the comments carefully, and don't do this in an image you care about!
"This creates a very dangerous object" Object subclass: #Nuke instanceVariableNames: 'one two three' classVariableNames: '' poolDictionaries: '' category: 'Test'. class := Smalltalk at: #Nuke. class superclass: nil. inst := class basicNew.
"This is harmless" MirrorInspector inspect: inst.
"This will crash your image" inst yourself.
The implementation uses primitive 188 to directly execute CompiledMethods with the reflected object as the receiver, thus giving us access to the object's state without sending it a message.
In part you can use the light-weigth mirror methods in ContextPart, these take the object operated on as an argument. Non-invasive printing could be done via execution simulation. I've already modified the Squeak debugger in Qwaq/Teleplace images to use these light-weight mirror primitives so that messages are not sent to receivers when simulating execution in the debugger (i.e. when doing send instead of step). If the mirror primitives now work on the interpreter I can fold this into trunk. David, do the mirror primitive tests pass on the interpreter?
All mirror primitive tests pass on the interpreter VM if built with the latest VMMaker and SVN sources. However, the mirror primitive support is not yet present in any officially released standard VM.
The updates for mirror primitive support were added in VMMaker-dtl.262 and VMMaker-dtl.261. Note the update comment in VMMaker-dtl.262, as the implementation differs from Cog and a code review would be welcome:
Name: VMMaker-dtl.262 Author: dtl Time: 5 January 2012, 12:41:29.745 am
VMMaker 4.7.19
Reference Mantis 7429: Add Mirror Primitives to the VM
Update primitivePerformInSuperclass to support ContextPart>>object:perform:withArguments:inClass:
Implementation differs from that of oscog in that the original primitivePerformAt: is retained unmodified, and the necessary stack adjustments are done in primitivePerformInSuperclass for the special case of argumentCount 4 (mirror primitive call) rather than 3. The oscog approach may be prefered (not least for its clearer method naming), but making the change in primitivePerformInSuperclass is low risk and more easily implemented by a Sunday Squeaker.
All MirrorPrimitiveTests pass.
Dave
squeak-dev@lists.squeakfoundation.org
-
Colin Putney -
David T. Lewis -
Eliot Miranda -
Frank Shearar -
Igor Stasenko