Hi All.
Thank you Chris for Magma, I am planning on using it on a commercial website. I will publish a link when it is ready.
Now for my challenge - Seaside support. The questions of course are open to all and comments are welcome. Thank you in advance. I've read all that I can on the subject but I still have a number of questions.
I am running the latest Pharo with the latest Magma loaded. All packages were loaded via Monticello.
1. The state of Jetsam. I installed the latest but now am unable to configure a seaside app. On the configuration page the options are "fixed set by client". Any suggestions? My concern -- is this is indicative of other problems -- maybe related to Pharo?
2. Magma SeasideHelper - Given the question about Jetsam this is a non-starter. I would assume SeasideHelper is of no use without Jetsam.
3. Magma Seaside - http://wiki.squeak.org/squeak/5817 - Can I use the examples and implement Magma as indicated? I am currently using a custom WASession for my purposes and will most likely establish a Magma connection for each Seaside session. Local database for development - remote for production.
4. Are there other implementation options - sample code? I understand the Gjallar implementation is not recommended - true?
Hi!
Timothy James Ziebart wrote:
- Are there other implementation options - sample code? I understand
the Gjallar implementation is not recommended - true?
If the question is - should I look at Gjallar for general information on how to use Magma (regardless of Seaside actually) then the answer is no. :)
But sure, if you take the time to understand *why* this is so, then there may be interesting bits in there still.
Gjallar uses the "Command pattern" extensively which means that all modifications to the object model are done by instantiating a certain "Transaction" class, feeding it with all state desribing the change to be made, creating a Magma transaction, telling this object to "apply" itself, add that object to a never ending MagmaCollection and finally committing.
This means Gjallar has both a traditional Magma domain model AND an endless collection of "delta" objects that describe each and every change that has been made to the model.
The reason was originally to support an offline mode, which still exists, but no user of Gjallar has yet actually made use of that mode.
regards, Göran
PS. Gjallar will soon move to latest Magma, and hopefully gain performance from that. We are also probably going to test other means of persistence during the autumn. Magma is great and cool, but we do have some performance problems in Gjallar and while I can't say 100% for sure it is all Magma related we will profile and experiment.
Thanks for the note Göran, I've been racked with curiosity myself about all the comments saying Gjaller "should not be used as an example of how to use Magma.."
But I am not much clearer about the why. Gjaller uses a common OO design-pattern, Command, so why is it a "bad example" of how to use (an ODBMS like) Magma?
Perhaps the real rumour is that it (Gjaller) is not a good example of how to integrate Magma with Seaside? If so, I'd like to hear more about the "why" of that.. Can you or someone just summarize what's being done, so we can all analyze and talk about it before we judge whether its "good" or "bad"?
You said:
This means Gjallar has both a traditional Magma domain model AND an endless collection of "delta" objects that describe each and every change that has been made to the model.
and this says, to me, that Magma should actually be fitting Gjaller like a glove, because most frameworks _either_ do well with large collections (i.e., RDBMS) but poorly with traditional object models (mapping), OR, they do well with traditional object models (ODBMS) but not as good with large-collections (BTree, RcBag, etc.). Magma has applied itself heavily to both problems.
What version of Magma are you running? As for performance, Magma came a long way in performance department in r41 and r42. If you still have performance "problems" in r42 then I'm _really_ curious to see the stats report from a troubled client session as well as the server. If you:
myMagmaSession statistics report
for the client, or, you can alternatively print it to any stream (if, for example the printString limit can sometimes be hit with these reports):
myMagmaSession statistics printReportTo: someStream
and post it up here (or send me private e-mail if it's too sensitive) it would reveal a lot about the _nature_ of Gjallers usage of Magma and potential problem areas..
If you are running an HA (which provides more scale, but necessarily more speed for individual clients), then please be sure to get stats from _both_ the primary and secondary:
myMagmaSession serverStatisticsPrimary report
and
myMagmaSession serverStatisticsSecondary report
Regards, Chris
2009/8/24 Göran Krampe goran@krampe.se:
Hi!
Timothy James Ziebart wrote:
- Are there other implementation options - sample code? I understand the
Gjallar implementation is not recommended - true?
If the question is - should I look at Gjallar for general information on how to use Magma (regardless of Seaside actually) then the answer is no. :)
But sure, if you take the time to understand *why* this is so, then there may be interesting bits in there still.
Gjallar uses the "Command pattern" extensively which means that all modifications to the object model are done by instantiating a certain "Transaction" class, feeding it with all state desribing the change to be made, creating a Magma transaction, telling this object to "apply" itself, add that object to a never ending MagmaCollection and finally committing.
This means Gjallar has both a traditional Magma domain model AND an endless collection of "delta" objects that describe each and every change that has been made to the model.
The reason was originally to support an offline mode, which still exists, but no user of Gjallar has yet actually made use of that mode.
regards, Göran
PS. Gjallar will soon move to latest Magma, and hopefully gain performance from that. We are also probably going to test other means of persistence during the autumn. Magma is great and cool, but we do have some performance problems in Gjallar and while I can't say 100% for sure it is all Magma related we will profile and experiment.
Magma mailing list Magma@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma
Hi!
Chris Muller wrote:
Thanks for the note Göran, I've been racked with curiosity myself about all the comments saying Gjaller "should not be used as an example of how to use Magma.."
But I am not much clearer about the why. Gjaller uses a common OO design-pattern, Command, so why is it a "bad example" of how to use (an ODBMS like) Magma?
Well, it would not be "typical" at least. "Bad" might have been an odd word. Let's say someone wants to learn how to use Magma, then they start reading the code in Gjallar and get quite confused about the seemingly huge amount of work being made just in order to change a silly field in an object.
A BlablaTxn class is instantiated, it gets fed some things, a Magma transaction is started, the txn object is sent #apply, yadda yadda. To a newbie that just wants to learn how "easy" it is to use an OODB like Magma it would look horrible.
Perhaps the real rumour is that it (Gjaller) is not a good example of how to integrate Magma with Seaside? If so, I'd like to hear more about the "why" of that..
Besides the fact that we have implemented our own Session pool (first we used a hacked one from Kilaua, then Keith changed it even more IIRC, then recently I reimplemented a new pool using queues instead to make it simpler and more robust - might be worth adopting into Magma, although I think I read you have one of your own now too) I don't think we do odd things regarding Seasde/Magma.
Earlier we played around a bit with sharing a readonly Magma session between Seaside sessions but we dropped that quite a while back when Keith managed to get some decent speed using read strategies.
Can you or someone just summarize what's being done, so we can all analyze and talk about it before we judge whether its "good" or "bad"?
I can do this but I really need more time and I need to look at the code to "remember" the details :)
You said:
This means Gjallar has both a traditional Magma domain model AND an endless collection of "delta" objects that describe each and every change that has been made to the model.
and this says, to me, that Magma should actually be fitting Gjaller like a glove, because most frameworks _either_ do well with large collections (i.e., RDBMS) but poorly with traditional object models (mapping), OR, they do well with traditional object models (ODBMS) but not as good with large-collections (BTree, RcBag, etc.). Magma has applied itself heavily to both problems.
Sure, I don't think I said Magma was a bad fit - I just meant that for any Magma/Seaside newbie that wants to build a "regular" Seaside/Magma app Gjallar displays some oddities that is not really representable - due especially to the "Command pattern" being used 100% for all db modifications.
What version of Magma are you running? As for performance, Magma came a long way in performance department in r41 and r42.
Yeah, we need to upgrade to the latest, we are using r42Alpha4 right now.
If you still have performance "problems" in r42 then I'm _really_ curious to see the stats report from a troubled client session as well as the server.
I will try to do that when I get back to Gjallar hacking. We have a customer that is complaining about the performance issues so I *will* need to look into this in the coming months anyway.
And I am also investigating alternative db solutions, perhaps mostly because I am interested in the subject - and of course, we really need better performance in Gjallar and while I can't say for sure that Magma is the culprit I need to have something to compare with. :) The top candidate that I am playing with right now is MongoDB. CouchDB doesn't fit Gjallar very well (poor dynamic query abilities) and TC/TT is too bare bones for Gjallar (and no community).
Another aspect is that we have experienced "odd problems" here and there and it makes me nervous for larger deployments. For example, indexing getting "messed up" (I really would like an ability to rebuild them easily) makeing query results "wrong". The "magic" of Magma is also a problem when something odd happens. It can be quite hard to know what to do to fix it.
Here is a short story explaining what I mean with this last bit:
I am helping a customer right now with building a system in C#. They used NHibernate+MSSQL and they got sick of hunting down problems they just couldn't understand. There was simply too much "magic" going on that wasn't stable enough or predictable enough.
I then helped them to move to CouchDB by implementing a C# binding for CouchDB. Sure, the persistence needs a bit more work, but it is not as complex as ORM mapping and above all - it is highly predictable and very little magic going on. And the db has a nice web GUI builtin so that I can directly inspect it and see what the hell I just put in there. ---
Anyway, I don't think we will ever stop having Magma as a db option in Gjallar - the "all Squeak" aspect is worth a lot and alternate db options will never offer the same degree of transparency that Magma/Gemstone offers. And our meta model is quite complex and would require lots of work in a non transparent db.
My opinion though is that if we can house multiple db options in the same Gjallar codebase - then that can never be a bad thing. Thus, the port to Gemstone seemed to be able to coexist nicely with Magma.
Anyway, I would like to end this long post by saying that I love OODBs and I love that Squeak has one of the most advanced open source OODB that I know of. I am amazed at ýour effort here
regards, Göran
Another aspect is that we have experienced "odd problems" here and there and it makes me nervous for larger deployments. For example, indexing getting "messed up" (I really would like an ability to rebuild them easily) makeing query results "wrong".
You've mentioned this before.. I hope to have a chance to learn more about this problem. Magma's indexing has been heavily tested, but that doesn't mean there couldn't be an odd bug somewhere..
The "magic" of Magma is also a problem when something odd happens. It can be quite hard to know what to do to fix it. .. Here is a short story explaining what I mean with this last bit: ...
I was hoping you were going to share a Magma story! For the last couple of years, Magma has been developed almost solely based on user-requests; even the HA function! The next time something odd happens, please share and Magma will be fixed / improved to avoid that oddity..
Regards, Chris
Hi!
Chris Muller wrote:
Another aspect is that we have experienced "odd problems" here and there and it makes me nervous for larger deployments. For example, indexing getting "messed up" (I really would like an ability to rebuild them easily) makeing query results "wrong".
You've mentioned this before.. I hope to have a chance to learn more about this problem. Magma's indexing has been heavily tested, but that doesn't mean there couldn't be an odd bug somewhere..
Well, I dunno, I presume that it can't really be an "app side" bug causing it - but hey. :)
The "magic" of Magma is also a problem when something odd happens. It can be quite hard to know what to do to fix it. .. Here is a short story explaining what I mean with this last bit: ...
I was hoping you were going to share a Magma story! For the last couple of years, Magma has been developed almost solely based on user-requests; even the HA function! The next time something odd happens, please share and Magma will be fixed / improved to avoid that oddity..
I will try to capture it (and all other issues we see) - the problem is that it can be very hard to know the "when and how" when I suddenly notice that a case (in Gjallar) appears in the "wrong" place.
regards, Göran
Well, I dunno, I presume that it can't really be an "app side" bug causing it - but hey. :)
Magma requires the app to #noteOldKeysFor: an object, *before* its keys are changed. So it could be an app-side bug, but it's possible there could be a Magma bug too..
You can use the MagmaBufferBrowser to find the buffer for the bad Case, from that the exact commit-log record that indexed that case in the place it did. Explore that commit-log record and drill into the commit-package. Under there you will see the 'largeCollectionChanges' and, under there, a Dictionary with a key to see all of the details; what hash-keys the object was indexed at for each index..
Hope that helps..
magma@lists.squeakfoundation.org