I've been watching the work on SPrevayler with a cocked eyebrow, because I have to admit that I find it an extremely clunky approach, at least in its Java incarnation. Stephen's suggestion to combine it with REPLServer is interesting, but I think goes in the wrong direction - we don't want to be logging doIts (isn't that what the .changes file does anyway?), and we don't really want to be logging commands, we want to be logging message sends.
There are a couple of steps that I think could be taken to make the Prevayler approach more "smalltalky" and more manageable. The first is to get rid of Command objects and replace them with Message objects. That is, wrap a proxy around the PrevaylerSystem that implements DNU to capture and log any messages sent to the system before forwarding them on (if you like, you can wrap these in a Command object before serializing them, but you'll only need that one class of Command). This gets rid of the ridiculous duplication of having to implement a Command class with
executeOn: aSystem aSystem someMessage: ...
for every message on System.
The second step is to maintain a weak dictionary mapping unique ids to any object that passes through this proxy as a parameter, so that these ids can be written into the log instead of the objects themselves - you only need to fully serialize the object into the log when the id is first assigned (and, of course, you should replace any objects that already have ids during this serialization, with an id marker).
At that point you'd have something halfway between Prevayler and an OODB - no transactions, still a single bottleneck for mutation, but much more flexibility in what your commands can look like and much less overhead in implementing them.
Marco, Stephen, thoughts?
Hi. Your post is interesting. What you think is good. The whole prevayler thing is very Java like because I come from java and because of this I can't often see all the great features of smalltalk.
Avi Bryant wrote:
I've been watching the work on SPrevayler with a cocked eyebrow, because I have to admit that I find it an extremely clunky approach, at least in its Java incarnation. Stephen's suggestion to combine it with REPLServer is interesting, but I think goes in the wrong direction - we don't want to be logging doIts (isn't that what the .changes file does anyway?), and we don't really want to be logging commands, we want to be logging message sends.
There are a couple of steps that I think could be taken to make the Prevayler approach more "smalltalky" and more manageable. The first is to get rid of Command objects and replace them with Message objects. That is, wrap a proxy around the PrevaylerSystem that implements DNU to capture and log any messages sent to the system before forwarding them on (if you like, you can wrap these in a Command object before serializing them, but you'll only need that one class of Command). This gets rid of the ridiculous duplication of having to implement a Command class with
executeOn: aSystem aSystem someMessage: ...
for every message on System.
You want me to create a porxy that stands before the system. The proxy gets all the messages that should be send to system and serializes these to disk. What you mean is just writing the Object>>perform: withArguments: to the log files? Do I have to put the logic into doesNotUnderstand: of the proxy class?
But in fact it stays the same. The prevalence layer is just used behind the scenes. If using the above the prevayler would write the system as usual just the commands are created hidden. We would have a command like PerformCommand that keeps track of the sent commands.
Sorry for my pie-sized brain but I'm just a newbie to squeak.
Regards Marco
The second step is to maintain a weak dictionary mapping unique ids to any object that passes through this proxy as a parameter, so that these ids can be written into the log instead of the objects themselves - you only need to fully serialize the object into the log when the id is first assigned (and, of course, you should replace any objects that already have ids during this serialization, with an id marker).
This is not very useful in my mind. The outputs I use have a built in mechnism for this. (
At that point you'd have something halfway between Prevayler and an OODB - no transactions, still a single bottleneck for mutation, but much more flexibility in what your commands can look like and much less overhead in implementing them.
Marco, Stephen, thoughts?
On Fri, 28 Feb 2003, Marco Paga wrote:
You want me to create a porxy that stands before the system. The proxy gets all the messages that should be send to system and serializes these to disk. What you mean is just writing the Object>>perform: withArguments: to the log files? Do I have to put the logic into doesNotUnderstand: of the proxy class?
Yes, it'll need to go in #doesNotUnderstand:.
But in fact it stays the same. The prevalence layer is just used behind the scenes. If using the above the prevayler would write the system as usual just the commands are created hidden. We would have a command like PerformCommand that keeps track of the sent commands.
That's right.
ids to any object that passes through this proxy as a parameter, so that these ids can be written into the log instead of the objects themselves - you only need to fully serialize the object into the log when the id is first assigned (and, of course, you should replace any objects that already have ids during this serialization, with an id marker).
This is not very useful in my mind. The outputs I use have a built in mechnism for this.
Hm. Do they? Something like ReferenceStream will maintain object identity during a single serialization, but I don't think in your current implementation you maintain identity across multiple commands. That is, if you save the same object as part of two commands, and you read them back from the log later, will the two commands you recreate actually have the same object, or two similar but different ones? I'm pretty sure it'll be the latter, and that's what I'm trying to avoid.
To put it another way, my understanding is that currently Command objects should only contain Strings, Numbers, etc, that can be easily serialized and restored. I'd like them to be able to contain any domain object, and this means assigning ids so you can maintain identity.
This might be too different from the Prevayler idea for you to want to implement it, but I think it would be a useful system - maybe I'll work up a prototype this weekend.
Avi
Hm. Do they?
Yes they do it, I know it because I had some problems with it because ot this feature.
Something like ReferenceStream will maintain object identity during a single serialization, but I don't think in your current implementation you maintain identity across multiple commands. That is, if you save the same object as part of two commands, and you read them back from the log later, will the two commands you recreate actually have the same object, or two similar but different ones? I'm pretty sure it'll be the latter, and that's what I'm trying to avoid.
To put it another way, my understanding is that currently Command objects should only contain Strings, Numbers, etc, that can be easily serialized and restored. I'd like them to be able to contain any domain object, and this means assigning ids so you can maintain identity.
This might be too different from the Prevayler idea for you to want to implement it, but I think it would be a useful system
If it is a good thing I will implement it. No question. But I think that it is very similar to my intention. For me it is very important to create a system that is useful not just another prevayler port.
- maybe I'll work up
a prototype this weekend.
That would be great. I would like to see it to better understand what you want to create.
Avi
Regards Marco
I wrote a little Proxy that does the things you wanted. Here an example:
| p c base proxy| base_ FileDirectory on: (FileDirectory default pathName , FileDirectory slash , 'testBase'). p := Prevayler withSystem: (MPTestSystem value: 0) on: base. proxy_ PrevalenceProxy prevayler: p. proxy addValue: 17.
every message that goes to the proxy is handeld by the prevalence layer. You have to pay attention in the case of creating a new prevayler because then you have to create a new proxy too.
Regards Marco
Avi Bryant wrote:
On Fri, 28 Feb 2003, Marco Paga wrote:
You want me to create a porxy that stands before the system. The proxy gets all the messages that should be send to system and serializes these to disk. What you mean is just writing the Object>>perform: withArguments: to the log files? Do I have to put the logic into doesNotUnderstand: of the proxy class?
Yes, it'll need to go in #doesNotUnderstand:.
But in fact it stays the same. The prevalence layer is just used behind the scenes. If using the above the prevayler would write the system as usual just the commands are created hidden. We would have a command like PerformCommand that keeps track of the sent commands.
That's right.
ids to any object that passes through this proxy as a parameter, so that these ids can be written into the log instead of the objects themselves - you only need to fully serialize the object into the log when the id is first assigned (and, of course, you should replace any objects that already have ids during this serialization, with an id marker).
This is not very useful in my mind. The outputs I use have a built in mechnism for this.
Hm. Do they? Something like ReferenceStream will maintain object identity during a single serialization, but I don't think in your current implementation you maintain identity across multiple commands. That is, if you save the same object as part of two commands, and you read them back from the log later, will the two commands you recreate actually have the same object, or two similar but different ones? I'm pretty sure it'll be the latter, and that's what I'm trying to avoid.
To put it another way, my understanding is that currently Command objects should only contain Strings, Numbers, etc, that can be easily serialized and restored. I'd like them to be able to contain any domain object, and this means assigning ids so you can maintain identity.
This might be too different from the Prevayler idea for you to want to implement it, but I think it would be a useful system - maybe I'll work up a prototype this weekend.
Avi
Hi Marco, Avi, Stephen... and all
From the view of usage of Prevayler (as it is implemented at the moment) I
see the two following shortcomings that need to be addressed:
1. Proxy-Solution
every message that goes to the proxy is handeld by the prevalence layer.
I think this is problematic because you log each message that is sent to the domain object, if it is relevant or not, making the log file much bigger than necessary. (With the current implementation you have to insert some return statements to be able to use the proxy to return objects other than self). So there should possibly be some kind of distinction if a method is effecting the state of the object or not!?
2. As it is now, you have to implement all your relevant methods in one main domain object (a PrevalentSystem). I think this will lead to a bad design of the domain model. It would be nice if it was possible to record all relevant messages to all domain objects.
What do you think? I'll implement some tests (a simple bank account model e.g.) where we can better test/show how to use Prevayler, ok?
Cheers, Adrian
_____________________ Adrian Lienhard www.adrian-lienhard.ch www.netstyle.ch
----- Original Message ----- From: "Marco Paga" mail@marco-paga.de To: "The general-purpose Squeak developers list" squeak-dev@lists.squeakfoundation.org Sent: Friday, February 28, 2003 1:48 PM Subject: Re: SPrevayler
I wrote a little Proxy that does the things you wanted. Here an example:
| p c base proxy| base_ FileDirectory on: (FileDirectory default pathName , FileDirectory slash , 'testBase'). p := Prevayler withSystem: (MPTestSystem value: 0) on: base. proxy_ PrevalenceProxy prevayler: p. proxy addValue: 17.
every message that goes to the proxy is handeld by the prevalence layer. You have to pay attention in the case of creating a new prevayler because then you have to create a new proxy too.
Regards Marco
Avi Bryant wrote:
On Fri, 28 Feb 2003, Marco Paga wrote:
You want me to create a porxy that stands before the system. The proxy gets all the messages that should be send to system and serializes these to disk. What you mean is just writing the Object>>perform: withArguments: to the log files? Do I have to put the logic into doesNotUnderstand: of the proxy class?
Yes, it'll need to go in #doesNotUnderstand:.
But in fact it stays the same. The prevalence layer is just used behind the scenes. If using the above the prevayler would write the system as usual just the commands are created hidden. We would have a command like PerformCommand that keeps track of the sent commands.
That's right.
ids to any object that passes through this proxy as a parameter, so
that
these ids can be written into the log instead of the objects
themselves -
you only need to fully serialize the object into the log when the id is first assigned (and, of course, you should replace any objects that already have ids during this serialization, with an id marker).
This is not very useful in my mind. The outputs I use have a built in mechnism for this.
Hm. Do they? Something like ReferenceStream will maintain object identity during a single serialization, but I don't think in your current implementation you maintain identity across multiple commands. That is, if you save the same object as part of two commands, and you read them back from the log later, will the two commands you recreate actually have the same object, or two similar but different ones? I'm pretty sure
it'll
be the latter, and that's what I'm trying to avoid.
To put it another way, my understanding is that currently Command objects should only contain Strings, Numbers, etc, that can be easily serialized and restored. I'd like them to be able to contain any domain object, and this means assigning ids so you can maintain identity.
This might be too different from the Prevayler idea for you to want to implement it, but I think it would be a useful system - maybe I'll work
up
a prototype this weekend.
Avi
On Sun, 2 Mar 2003, Adrian Lienhard wrote:
- Proxy-Solution
every message that goes to the proxy is handeld by the prevalence layer.
I think this is problematic because you log each message that is sent to the domain object, if it is relevant or not, making the log file much bigger than necessary. (With the current implementation you have to insert some return statements to be able to use the proxy to return objects other than self). So there should possibly be some kind of distinction if a method is effecting the state of the object or not!?
- As it is now, you have to implement all your relevant methods in one main
domain object (a PrevalentSystem). I think this will lead to a bad design of the domain model. It would be nice if it was possible to record all relevant messages to all domain objects.
Adrian,
As I see it only one of your points can be relevant at a time. If, as is the general idea with Prevayler, all mutation has to go through a central "PrevalentSystem" object, then your first point doesn't hold - simple queries can be implemented directly on the domain objects, with only state-affecting actions having to go through the PrevalentSystem and be logged.
If, as you suggest in your second point, we log message sends to all domain objects, then we do indeed need some way to tell which methods affect state and which do not (we also definitely need to assign unique ids to these objects, so that the logged messages have a meaningful receiver). This is a much more complicated solution, however. At that point I think I would rather log the mutations themselves (all inst var writes), which would be possible with a modified VM like Stephen's Chango.
Just a crazy thought off the top of my head - if you did want to widen the bottleneck a little bit, and allow arbitrary methods on your domain objects to be marked as mutators, you could introduce an AboutToModifyStateNotification, then use a handler that would log the receiver, message, and arguments of the method context from which it was signaled. Ideally perhaps the notification would take a block with the state-affecting code, so that it could ignore recursive invocations.
Avi
Hi Avi
As I see it only one of your points can be relevant at a time. If, as is the general idea with Prevayler, all mutation has to go through a central "PrevalentSystem" object, then your first point doesn't hold - simple queries can be implemented directly on the domain objects, with only state-affecting actions having to go through the PrevalentSystem and be logged.
Yes, but you may also have methods in your central System that don't have to be logged. Let's take the bank account example. I would implement the "bank" as the central class which handels all relevant methods to log. Bank holds customers and accounts. But now, if you wrap the bank with the proxy, you also log simple getter messages like #accounts...
If, as you suggest in your second point, we log message sends to all domain objects, then we do indeed need some way to tell which methods affect state and which do not (we also definitely need to assign unique ids to these objects, so that the logged messages have a meaningful receiver). This is a much more complicated solution, however. At that point I think I would rather log the mutations themselves (all inst var writes), which would be possible with a modified VM like Stephen's Chango.
Yes, this doesn't seem to be a simple thing. It was more like a wish, which would make it an almost perfect transparent persistence framework. I can live very well with what it can do know.
Cheers, Adrian
Just a crazy thought off the top of my head - if you did want to widen the bottleneck a little bit, and allow arbitrary methods on your domain objects to be marked as mutators, you could introduce an AboutToModifyStateNotification, then use a handler that would log the receiver, message, and arguments of the method context from which it was signaled. Ideally perhaps the notification would take a block with the state-affecting code, so that it could ignore recursive invocations.
Avi
Hi.
Adrian Lienhard wrote:
Hi Avi
As I see it only one of your points can be relevant at a time. If, as is the general idea with Prevayler, all mutation has to go through a central "PrevalentSystem" object,
The queries to the domain object (PrevalentSystem) go through the Prevayler.
then your first point doesn't hold - simple queries can be implemented directly on the domain objects, with only state-affecting actions having to go through the PrevalentSystem and be logged.
Yes, but you may also have methods in your central System that don't have to be logged.
You can use PrevaleceProxy>>system method to query the state. But if you want to alter something you have to use the proxy.
Let's take the bank account example. I would implement the "bank" as the central class which handels all relevant methods to log. Bank holds customers and accounts. But now, if you wrap the bank with the proxy, you also log simple getter messages like #accounts...
If, as you suggest in your second point, we log message sends to all domain objects, then we do indeed need some way to tell which methods affect state and which do not (we also definitely need to assign unique ids to these objects, so that the logged messages have a meaningful receiver). This is a much more complicated solution, however. At that point I think I would rather log the mutations themselves (all inst var writes), which would be possible with a modified VM like Stephen's
I can follow you to the point until you go into the instVars and so on. It would be to detailed. I think that we have to see it a little bit more abstract. I think we have to go away from details and think about how we could use the architecture of prevayler in another way to do what we want. At the moment I'm thinking about something that could give the trick. But it is to abstract to tell (Sorry - chaos in my brain ;-) ) At the moment I really don't know what the right way for SPrevayler is and it is a long way to go.
Chango. Yes, this doesn't seem to be a simple thing. It was more like a wish, which would make it an almost perfect transparent persistence framework. I can live very well with what it can do know.
Cheers, Adrian
Just a crazy thought off the top of my head - if you did want to widen the bottleneck a little bit, and allow arbitrary methods on your domain objects to be marked as mutators, you could introduce an AboutToModifyStateNotification, then use a handler that would log the receiver, message, and arguments of the method context from which it was signaled. Ideally perhaps the notification would take a block with the state-affecting code, so that it could ignore recursive invocations.
Avi
In my mind that is something that is against the idea of prevalence. The programmer would have to think on more details. SPrevayler would become feature laden and that is something I don't want prevayler to be. In my mind it should be easy to use. You should not see a difference between code that uses prevayler and code that doesn't use prevayler. I don't see a point for that at the moment. If there is a day when I think for this feature we need to code it that way ok but at the moment I'm not convinced. There is so much that can be handled with a clean design without manifesting the presistence layer in the code.
Regards
Marco
Hi guys.
Adrian Lienhard wrote:
Hi Marco, Avi, Stephen... and all
From the view of usage of Prevayler (as it is implemented at the moment) I
see the two following shortcomings that need to be addressed:
- Proxy-Solution
every message that goes to the proxy is handeld by the prevalence layer.
I think this is problematic because you log each message that is sent to the domain object, if it is relevant or not, making the log file much bigger than necessary. (With the current implementation you have to insert some return statements to be able to use the proxy to return objects other than self). So there should possibly be some kind of distinction if a method is effecting the state of the object or not!?
- As it is now, you have to implement all your relevant methods in one main
domain object (a PrevalentSystem). I think this will lead to a bad design of the domain model. It would be nice if it was possible to record all relevant messages to all domain objects.
A simple solution to this problem would be to have the logic distributed as usual through the whole domain objects and just some caller methods ( very general and light wieght ) in the PrevalenceSystem implemented. We would not have to hack the whole VM for this approch, but of coure this is just possible with a good design. But perhaps we can use the prevayler more in the background. As I saw while thinking about the proxy you can hide the persistence logic away (as much as possible) and do more things without the developer even noticing it. I don't know if there is a way that domain objects call the prevayler and tell him about a message they recieved that is about to change their state. Perhaps it could be done with AspectS and implemented as an aspect (There is a way in java to do this). Or is there a way in "clean" squeaky smalltalk? If we could do it that way we could hide the PrevalentSystem nearly fully away... That would be a persistence mechnism that I would like to use. If we achieved this that would be great. After that there are so many things that need to be done.
Regards Marco
On Sun, 2 Mar 2003, Marco Paga wrote:
and tell him about a message they recieved that is about to change their state. Perhaps it could be done with AspectS and implemented as an aspect (There is a way in java to do this). Or is there a way in "clean" squeaky smalltalk?
This was exactly what I was getting at with some kind of AboutToChangeNotification. Of course we could wrap it in a method on object so that this looked like
self changing: [ ... do something that changes state... ]
Avi Bryant wrote:
This was exactly what I was getting at with some kind of AboutToChangeNotification. Of course we could wrap it in a method on object so that this looked like
You want to take part in the project? That's great. Right at the moment we could just have some time to think about a clean design. I'm not really happy with what is going through my head. There must be more magic behind SPrevayler. Perhaps we have to think about where it should be headed. In my mind it should become a very light-weigth and very fault tolerant persistence layer. I don't know if it would be useful to have more in the api. There are enough other projects that can give you such facilities.
Regards Marco
self changing: [ ... do something that changes state... ]
Avi Bryant wrote:
I've been watching the work on SPrevayler with a cocked eyebrow, because I have to admit that I find it an extremely clunky approach, at least in its Java incarnation. Stephen's suggestion to combine it with REPLServer is interesting, but I think goes in the wrong direction - we don't want to be logging doIts (isn't that what the .changes file does anyway?), and we don't really want to be logging commands, we want to be logging message sends.
[snip]
At that point you'd have something halfway between Prevayler and an OODB - no transactions, still a single bottleneck for mutation, but much more flexibility in what your commands can look like and much less overhead in implementing them.
Marco, Stephen, thoughts?
My comments were based solely on having read the docs (I haven't had the time to actually load the code). So, I'm sure that your critique of the implementation is completely valid.
What I was getting at was a deployment scenario that would be easily manageable. I've found (with my Chango work) that you need a very clean separation between what objects comprise your "application" and which ones are "data" (aka domain). The temporal evolution requirements of these two kinds of objects are very different (note: this discussion is very much related to the one last weekend on imperative vs declarative program construction).
Now, having said this, I do realize that in an ideal world, we'd have a sufficiently robust system that would be capable of accomodating both sets of needs in a single unified approach. But, I'm speaking about the here and now, not the ideal world of the future.
So, if I wanted to deploy a real world systems using a prevaylor style approach (which by the way, I've seen commercial Smalltalk systems that use this very approach), this is probably how I would do it:
The transactional requirements for an application are as follows: we need to start from some well known base; the base VM and image. We need to apply transitions to the system that add our application code to that base (loading change sets, SARs, DVS packages, etc). In this way, we have controlled approach to evolving from a base Squeak image to an image that is designed to host our application (the application may or may not need to include the meta-model for our data objects). Sounds very much like a traditional build process right?
The transactional requirements for our data is as follows: we need to start from some well know base; we need to install the meta-model (aka "schema") for our data objects, we need to seed our database with initial state, we need to capture any instructions to tranform our data, we need to capture any instructions that transform our data's meta-model. We need to log all such instructions to disk for replay in the event that the system goes down. On start up, we must first replay all logged instructions (that we logged since the last checkpoint) and probably checkpoint (snapshot) the system. We also will want to periodically checkpoint the system. Sounds very much like a traditional RDBMS right?
At runtime, I would have one Squeak process running my application, and another hosting my data. Communications between the two would happen through something like REPLServer that would essientially enable me to do anything to the data image (including applying changes to the meta-model). These commands are the ones that would be logged (and your data image would not refer to the clock or any other external resource directly, instead it would receive any such information through the REPL interface).
In this scenario, I'm separating the application and data objects (and data meta-model) into two distinct images based on their differing requirements in terms of managing temporal evolution. To deliver an upgrade of this system, I would deliver a complete new build of the application image, along with an upgrade script that applies a sequence of commands (through the REPL interface) to the already deployed "data" image. That upgrade script might apply changes to the meta-model, upgrade existing data objects to a new layout, etc.
On the application image side, in order to keep things more "smalltalky", I would use remote proxy objects to stand in place of real domain objects that dispatch REPLServer commands to the data image upon receiving messages. On top of this basic interface, I would begin to evolve a more sophisticated ORB like ability that managed remote object references, etc. In fact, this dual image approach would look very much like many of the GemStone deployment scenarios (where you have VW or VA connected to a GS server).
- Stephen
squeak-dev@lists.squeakfoundation.org
-
Adrian Lienhard -
Avi Bryant -
Marco Paga -
Stephen Pair