Hi there,
since nobody at ESUG seemed to be interested in a Glorp/Seaside BOF as I suggested, I try something new now ;-)
Glorp is a great tool for OR mapping and works very well. There is good introductory material, even if it is hard to find. But when it comes to real world use, there are many questions to be answered like
* how to handle transactions, mementos, units of work and that stuff, especially in a multi-user environment like a Seaside based web app * how to map more complicated things, like polymorphic joins, embedded objects etc. * what if an image slows down gradually when a user has been working for hours, almsot coming to a standstill. How to find out if Glorp or your use of it is the problem? What can you do then? * Is my use of transactions correct or will I face problems
None of these are easy questions and maybe there are many possible answers to them, But finding them on your own can be hard and take a lot of time. It can even be a risk to your business.
There is the Glorp group, but it's not actually in heavy use and sometimes it is hard to write down a problem in a few lines. Explaining the whole thing sometimes would make for loooong messages that probably nobody ever reads, esp. if the poster's english isn't brilliant.
So what I'm looking for is ways to find and meet people who also use Glorp (or any other ORM in Smalltalk) and would be interested in discussing stuff and maybe answer questions. Something like a Glorp users group.
The best thing to exchange ideas is probably to meet face to face and carry your laptop with you. So if anybody in Southern Germany, Eastern France, Northern Switzerland would be interested in such a meeting, please let me know and I am more than interested in setting something up.
The second best is probably some sort of online conference call. Maybe even on a regular basis.
Please let me know if you'd be interested.
Joachim
2017-09-28 3:56 GMT-03:00 jtuchel@objektfabrik.de jtuchel@objektfabrik.de:
Hi there,
Glorp is a great tool for OR mapping and works very well. There is good introductory material, even if it is hard to find. But when it comes to real world use, there are many questions to be answered like
how to handle transactions, mementos, units of work and that stuff, especially in a multi-user environment like a Seaside based web app how to map more complicated things, like polymorphic joins, embedded objects etc.
For that purpose, and in the context of Magritte, I created a MAEventedMemento, which basically redefines default #commit, and is evented in a three phase commit: as in #aboutToCommit, #commit, #postCommit, then in my magritte based editors I hook into these events as:
onAboutToCommit: anObject self app databaseSession hasUnitOfWork ifFalse: [ self app beginUnitOfWork ]. self app register: anObject
I use these in the context of a nested tree of objects where you can "dive in" to edit references, or add elements to a collection.
It is not recommended to keep transactions open for a long time because you have to handle the locking and other side-effects of doing it, but in my use case it was simpler to handle that than to implement a whole object graph buffering.
what if an image slows down gradually when a user has been working for hours, almsot coming to a standstill. How to find out if Glorp or your use of it is the problem? What can you do then? Is my use of transactions correct or will I face problems
I never had that behavior, since the Glorp session was tied to a Seaside session, and the Seaside session ended at some point. I'm talking about months of uptime with a small set of Pharo images handling the app.
The alternative is to use different types of caching policy in your Glorp session, but at that time Pharo's support of Ephemerons and other weak references wasn't clear, so I decided to not explore that path.
I know you use VAST, I'm sure its support for different caching policies is better.
None of these are easy questions and maybe there are many possible answers to them, But finding them on your own can be hard and take a lot of time. It can even be a risk to your business.
As it said in the Buddhist temple: "A hundred monks, a hundred religions"
So what I'm looking for is ways to find and meet people who also use Glorp (or any other ORM in Smalltalk) and would be interested in discussing stuff and maybe answer questions. Something like a Glorp users group.
I'm not actively using Glorp lately (only marginally in one project), but I'm fond to ORMs, I coded a complete ORM for my previous company (still being used) and I like SQL.
The best thing to exchange ideas is probably to meet face to face and carry your laptop with you. So if anybody in Southern Germany, Eastern France, Northern Switzerland would be interested in such a meeting, please let me know and I am more than interested in setting something up.
That's a little far from where I regularly am, but as I told you before, I'll be in Europe during November, and I'm willing to travel for one or two days to Switzerland.
The second best is probably some sort of online conference call. Maybe even on a regular basis.
I could certainly attend to these more often. :)
Regards!
Esteban A. Maringolo
Hi
Let me start by mentioning that I don't believe any of these issues are Seaside specific. IMO these issues apply to any web framework using GLORP and even any web framework using an ORM in general. So it could be beneficial to investigate how other Smalltalk web frameworks (AIDA, ApeX, Iliad, ...) or even other languages solve this issue (JSF, Spring Web MVC, JPA/Hibernate).
What is most concerning to me is how every object ever loaded with GLOP is tracked in the cache of the GLOP session. IMO this is wrong as most objects are read-only and never edited. These objects should not be tracked. This leads to a lot of unnecessary memory overhead. Only the objects that are actually edited should be tracked. I think of it this way, when a table or list of objects is rendered none the displayed objects should be tracked. Only when an edit link or button is clicked and an edit form is rendered then the edited object needs to be tracked and optimistically locked. If we track every object ever loaded then this simply leads to unbound memory usage. If caching has to happen for correctness then the cache should be flushed at the end of a request.
Digging a bit into the GLROP documentation I don't think a unit of work should be active during the render phase. As far as I understand this should prevent any of the loaded objects from being tracked by being added to the cache. Only when an object is being edited should it be registered manually for tracking by the GLOP session. However during the callback phase a unit of work should be active. This should be done automatically by a Seaside-GLORP extension. On a successful or cancelled edit the edited object should be removed from the cache again and the callback should be invalidated (ideally a one time callback is used).
It would be good to hear from somebody with GLROP experience whether what I just wrote made any sense at all or is utter nonsense.
I would also be open to attending some sort of meetup in northern Switzerland/southern Germany to discuss this in person.
Cheers Philippe
On Thu, Sep 28, 2017 at 8:56 AM, jtuchel@objektfabrik.de jtuchel@objektfabrik.de wrote:
Hi there,
since nobody at ESUG seemed to be interested in a Glorp/Seaside BOF as I suggested, I try something new now ;-)
Glorp is a great tool for OR mapping and works very well. There is good introductory material, even if it is hard to find. But when it comes to real world use, there are many questions to be answered like
how to handle transactions, mementos, units of work and that stuff, especially in a multi-user environment like a Seaside based web app how to map more complicated things, like polymorphic joins, embedded objects etc. what if an image slows down gradually when a user has been working for hours, almsot coming to a standstill. How to find out if Glorp or your use of it is the problem? What can you do then? Is my use of transactions correct or will I face problems
None of these are easy questions and maybe there are many possible answers to them, But finding them on your own can be hard and take a lot of time. It can even be a risk to your business.
There is the Glorp group, but it's not actually in heavy use and sometimes it is hard to write down a problem in a few lines. Explaining the whole thing sometimes would make for loooong messages that probably nobody ever reads, esp. if the poster's english isn't brilliant.
So what I'm looking for is ways to find and meet people who also use Glorp (or any other ORM in Smalltalk) and would be interested in discussing stuff and maybe answer questions. Something like a Glorp users group.
The best thing to exchange ideas is probably to meet face to face and carry your laptop with you. So if anybody in Southern Germany, Eastern France, Northern Switzerland would be interested in such a meeting, please let me know and I am more than interested in setting something up.
The second best is probably some sort of online conference call. Maybe even on a regular basis.
Please let me know if you'd be interested.
Joachim
--
Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
seaside mailing list seaside@lists.squeakfoundation.org http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
2017-10-01 16:40 GMT-03:00 Philippe Marschall philippe.marschall@gmail.com:
Hi
Let me start by mentioning that I don't believe any of these issues are Seaside specific. IMO these issues apply to any web framework using GLORP and even any web framework using an ORM in general. So it could be beneficial to investigate how other Smalltalk web frameworks (AIDA, ApeX, Iliad, ...) or even other languages solve this issue (JSF, Spring Web MVC, JPA/Hibernate).
I agree, in general.
The big difference is that Seaside works stateful, more like an application server than a single request-process-response execution.. I don't know much about Spring or other Java based tecnologies.
What is most concerning to me is how every object ever loaded with GLOP is tracked in the cache of the GLOP session. IMO this is wrong as most objects are read-only and never edited. These objects should not be tracked. This leads to a lot of unnecessary memory overhead. Only the objects that are actually edited should be tracked. I think of it this way, when a table or list of objects is rendered none the displayed objects should be tracked. Only when an edit link or button is clicked and an edit form is rendered then the edited object needs to be tracked and optimistically locked. If we track every object ever loaded then this simply leads to unbound memory usage. If caching has to happen for correctness then the cache should be flushed at the end of a request.
I'm not sure what you mean by "tracked"; one thing is the caching of such objects, and a different one is the rowmaps that GLORP keeps to detect modifications.
When GLORP reads an object from the database it builds a rowmap, which basically is the array of "database data" used to build the object. I agree that if we could detect or deterministically define the classes whose instances will not be modified, we could discard the rowmaps once the instance has been created, saving some memory if the read objects are many.
A different thing is the cache, which should be defined per class, so a "Person" could be kept in the cache (weak, timed), but a "PurchaseOrder" don't, however that is more a domain logic decision. Because if you repeatedly access the PurchaseOrder, the I/O time to fetch the data and build the object is an order of magnitude bigger than many computations.
Digging a bit into the GLROP documentation I don't think a unit of work should be active during the render phase. As far as I understand this should prevent any of the loaded objects from being tracked by being added to the cache. Only when an object is being edited should it be registered manually for tracking by the GLOP session.
You could work without a UnitOfWork (UoW) and only explicitly mark (register) in the session which objects are going to be "tracked" for persistance. What UoW does is to simplify that process by automatically registering all objects read whitin the UoW.
I don't tie the UoW with the render cycle, the UoW is, in my case, tied to a "single business logic unit" (hence, a UoW). I load the objects without a UoW, however most of these objects are going to be kept in the cache. In some cases I perform the queries explicitly requesting "fresh" versions, thus avoiding the cache hit.
If you're working with thousands of objects in a table (to use as simple example), you could paginate those, the caching of the previously queried pages will be determined by the caching policy or the explicit flushes you performed along the way.
However during the callback phase a unit of work should be active. This should be done automatically by a Seaside-GLORP extension. On a successful or cancelled edit the edited object should be removed from the cache again and the callback should be invalidated (ideally a one time callback is used).
In the callback phase you could need more than one UoW, that depends on how you structure your code.
I wouldn't use a UoW defined elsewhere. Maybe you're thinking in what GemStone does with the abort/commit transaction at every cycle.
It would be good to hear from somebody with GLROP experience whether what I just wrote made any sense at all or is utter nonsense.
What I don't understand is the need to remove everything from the cache in the context of a stateful application. The purpose of the cache is exactly to save I/O time, which as said, in the context of RDBMS can be the slowest of all.
The fine tune of the cache is, as with any other performance tuning, dependent on your business model.
I would also be open to attending some sort of meetup in northern Switzerland/southern Germany to discuss this in person.
Great!
Regards!
-- Esteban.
seaside@lists.squeakfoundation.org
-
Esteban A. Maringolo -
jtuchel＠objektfabrik.de -
Philippe Marschall