Magma September 2007

magma@lists.squeakfoundation.org

12 participants
20 discussions

Fwd: MagmaCollectionReader>>size takea a long time
by Burella Juan M. 05 Sep '07

05 Sep '07

Chris thanks for your answers. Yes, I was using the release 39. I´ve tested it with r40 (r40Gamma3) and it worked better. I tested this with a bigger collection (84661 objects ) and it takes 16 seconds to answer either #size and #lastKnownSize. Do I have to set something to enable #lastKnownSize caching? The new results are: "------------ 84661 objects - MagmaCollection>>size - fiirst evaluation" [ (magmaSession root at: #ThesaurusTerm) size ] timeToRun "answer about 247 milliseconds" "Note: magmaSession root at: #ThesaurusTerm answer a MagmaCollection" "second evaluation" [ (magmaSession root at: #ThesaurusTerm) size ] timeToRun "answer about 0 milliseconds" "MagmaCollectionReader>>size first evaluation" [ ((magmaSession root at: #ThesaurusTerm) read: #issnString) size ] timeToRun "answer about 16771 milliseconds" "second evaluation" [ ((magmaSession root at: #ThesaurusTerm) read: #issnString) size ] timeToRun "answer about 16870 milliseconds" "MagmaCollectionReader>>size first evaluation" [ ((magmaSession root at: #ThesaurusTerm) read: #issnString) lastKnownSize] timeToRun "answer about 16573 milliseconds" "second evaluation" [ ((magmaSession root at: #ThesaurusTerm) read: #issnString) lastKnownSize] timeToRun "answer about 16529 milliseconds" "------- 5500 objects - MagmaCollection>>size - fiirst evaluation" [ (magmaSession root at: #Serial) size ] timeToRun "answer about 698 milliseconds" "Note: magmaSession root at: #Serial answer a MagmaCollection" "second evaluation" [ (magmaSession root at: #Serial) size ] timeToRun "answer about 0 milliseconds" "MagmaCollectionReader>>size first evaluation" [ ((magmaSession root at: #Serial) read: #issnString) size ] timeToRun "answer about 1246 milliseconds" "second evaluation" [ ((magmaSession root at: #Serial) read: #issnString) size ] timeToRun "answer about 1246 milliseconds" Juan M. On 8/31/07, Chris Muller <asqueaker(a)gmail.com> wrote: > > Juan, please the section "Optimizing Performance" in the documentation > about querying MagmaCollections to understand when #size will be very > fast vs. pretty slow. > > http://wiki.squeak.org/squeak/5859 > > For the cases that are "pretty slow" (i.e., a Reader requiring > distinct results), note you have #lastKnownSize, which is the cached > size (therefore instant access). > > - Chris >

2 1

Re: Re: Removing References to Morphic
by "S.J.Chun" 04 Sep '07

04 Sep '07

OK, I'll try with latest version. And what I mean by "minimal image" is without morphic and mvc, not with morphic. Last time I tried to install Magma but there's many errors of undefined class(morphic related). Thank you and I'll report my result and problems(if any). ----- Original Message ----- From: Chris Muller <asqueaker(a)gmail.com> To: "\"S.J.Chun\"" <chunsj(a)embian.com> Cc: magma <magma(a)lists.squeakfoundation.org> Sent: 07-09-04 00:45:28 Subject: Re: Re: Removing References to Morphic The only reference to a Morph class that I know of are in MagmaSession>>#initializeSerializer. This is a soft reference to the Symbol, #Morph so that those serializing live Morph objects can do so transparently within the constraints presented by the framework. It should not present a problem in a minimal image with Morphic. This is the only reference I know of. Are you speaking of a different one? On 9/2/07, "S.J.Chun" <chunsj(a)embian.com> wrote: > Hi, > > Old seaside also did have references to Morphic classes and thiss was > really annoying to me like MVC or minimal image users. And from my > experience(maybe 3 or 4 months ago), Magma does have this problem. > > Can you remove/separate references to morphic related classes? > > TIA. > _______________________________________________ > Magma mailing list > Magma(a)lists.squeakfoundation.org > http://lists.squeakfoundation.org/mailman/listinfo/magma > >

1 0

Removing References to Morphic
by "S.J.Chun" 03 Sep '07

03 Sep '07

Hi, Old seaside also did have references to Morphic classes and thiss was really annoying to me like MVC or minimal image users. And from my experience(maybe 3 or 4 months ago), Magma does have this problem. Can you remove/separate references to morphic related classes? TIA.

2 1

Fwd: Magma
by Chris Muller 02 Sep '07

02 Sep '07

Hi Ramiro, I hope you don't mind my copying the Magma list since this is a useful discussion for everyone. == Scalability == In the context of web applications there are two tiers of scaling to more users. First, use multiple client sessions across different images all accessing a server in its own image. Second, serve the domain from multiple Magma servers. Let me explain in order. -- Tier 1, Multiple clients -- First, you configure to run multiple MagmaClient sessions. What you do is have each web session have its own MagmaSession view of the repository, and use ReadStrategy's and #stubOut: to keep only enough in memory for the current working web page. This was proposed years ago here: http://wiki.squeak.org/squeak/5817 and I think Brent may have successfully implemented the multi-image configuration described on that page under "Remote Connections". Goran and I have discussed it in the past, but I don't know whether he ever tried it in earnest. If so, I hope he will comment about this. That is the first tier. Before describing the second tier, let me first remind that the design of Magma is such that the client does 80% of the heavy-lifting; a remote server will mostly sit idle, starving for requests to process until you get around 10 (or more, depending on lots of factors), busy clients. Until then, it doesn't do any good to do move to the second tier (multiple servers) until your clients can swamp the remote server, causing it to begin to queue the requests it receives from the multiple client sessions, making them wait. -- Tier 2, multiple servers -- When you get to this point, you can then begin to break up the server duties across multiple repositories in several ways. 1) Partition your domain by its major categories and use MagmaForwardingProxy's to link to objects across repositories. For more information: http://wiki.squeak.org/squeak/5604 I think this is a smart thing to do from the beginning simply for organizational purposes, but it also helps performance by dividing up the server burden across multiple images/computers. This feature works and has been around for years, although I don't know anyone who has used it (because no one has ever asked about it). 2) Many domains have information that becomes immutable after it is created. For example, transactions in an accounting system, messages in a communication system, digitized external artifacts, etc. Making one or more separate "warm backups" provides a mostly up-to-date copy of your repository that are on-line for, for example, user queries and reporiting. This can alleviate considerable burden from the main repository, which is concerned with the still-dynamic portion of the persistent model. For more information: http://wiki.squeak.org/squeak/5724 == Faster? == Where did you hear the new version is much faster? Perhaps you are referring to Sig's new Dictionary's that sped things up? That may be the case but I haven't had time to look at them yet. Thanks to Sig, Magma now includes the ability to specify whatever WeakIdentityKeyDictionary you want (see MagmaPreferences), so you can try out some other Dictionary's but I can't promise I can support them (or a Magma implementation that uses them). They need to be tested. The first step would be to run the entire test suite with the replacement Dictionary(s). If those pass then you can be pretty confident, but I would burn them in in a heavily-used test environment for a good month first. Beyond this, Magma r40 is not any faster than 39, except for single-clause query's, which are 100X faster due to a bug fix. == 3.10 == The official 3.10 is supported, but I have not tried the 3.10 "Development image" yet. Unless Damien has changed low-level stuff there shouldn't be any problem. Again, the way to answer this question for yourself is to do what I would do to give you the answer; run the MagmaTester test suite. Documentation on how to do this is spelled out on the Magma home page on the Swiki. http://wiki.squeak.org/squeak/2661 I'll bet it will be fine, but if you encounter a problem, please let me know because I do intend to support 3.10. - Chris - Hide quoted text - On 8/29/07, Ramiro Diaz Trepat <ramirodt(a)gmail.com> wrote: > > > ---------- Forwarded message ---------- > From: Ramiro Diaz Trepat <ramirodt(a)gmail.com> > Date: Aug 28, 2007 6:24 PM > Subject: Magma > To: Chris Müller <chris(a)funkyobjects.org> > > Hello Chris, > I am happy you are about to release a new Magma. > I wanted to ask you a couple of questions. Is there a way to make > Magma scale, on a cluster with different servers? In case of course > one had a lot of queries from a very active web site for instance. Is > there a way to have several servers running on the same database files > or with replication? > Will it run on Damien Cassous' version of Squeak 3.10? > I've heared that the new version is much faster, do you have some > figures? > Thanks >

5 5

Old and unused objects in the repository (objects file)
by Robert Schwarz 02 Sep '07

02 Sep '07

Hi, I'm writing a small Seaside/Magma application and I'm using OrderedCollection and MagmaCollections. The application works fine. But when I take a look in the objective- file with a text editor I noticed that a removed object remains in the objective-file. Here are some code snippets for adding an object: issue ifNotNil: [self session commit: [self model add: issue]] and for removing: remove: anIssueObject self session commit: [self model remove: anIssueObject] Is this normal or do I anything wrong? Can I manually or automatic remove/collect old and unused objects? Kind regards Robert

2 1

Running Magma across different Squeak versions
by Chris Muller 02 Sep '07

02 Sep '07

Magma has very little special-case code for its code-base to maintain compatibility across a variety of Squeak images; 3.7, 3.8, 3.9, Croquet, base Tweak, squeak-dev and soon (if not already) 3.10. Yesterday the Gjaller team contacted me about a function they needed from Magma; generation of a continuously-increasing sequence number with no chance for a duplicate between any two clients. Today I augmented Magma with a scary solution to the problem, MagmaSession>>#executeInServer: aBlock. I've said there are caveats to this and now I would like to elaborate. To make this work I had to add the following classes to the #protocol (which you can observe by browsing the changes between Magma server-cmm.229 and Magma server-cmm.230). "Classes for serializing the codeBlock of a MaServerExecuteRequest" {MaBlockContextStorage . MaMethodContextStorage . MaCompiledMethodStorage . DiskProxy . MethodProperties . BlockContext . ContextPart . InstructionStream} Magma has enjoyed its wonderful compatibility and ease of migration thanks to staying mostly above the meta-layer. But you can see that by introducing these to the protocol in this way, we have already broken compatibility with 3.7 and 3.8, which do not include MethodProperties. I then wondered whether a Magma server on any squeak version could host a variety of clients of different versions. I tested it as soon as I got home, here's what I found: - There appears to be problems going between 3.7 and >3.7 images due to the introduction of ByteString in place of String. - 3.8 and 3.9 seem to work together, except for CompiledMethods due to, once again, the aforementioned MethodProperties. If you are still with me, thank you. The point of all this is I would like to know how others feel about: 1) 3.7 compatibiltiiy at all 2) mixing 3.8 and >3.8 images 3) the desire to use Blocks, CompiledMethods or Processes in your persistent model 4) the risks of adding all those honkin' classes above to the protocol just to support executeInServer:. Thanks..

3 2

Release 40 is imminent
by Chris Muller 01 Sep '07

01 Sep '07

I have just posted r40Gamma3 to the "MagmaTester" project of squeaksource. This will be released to SqueakMap with an announcement sometime next week or so. This one single code-base has been tested successfully in 3.7, 3.8, 3.9, Croquet, base Tweak and the squeak-Development images. Obviously, the one missing from that list is 3.10, simply because I haven't had time to do it yet. That's the final gate to pass thru before releasing. This is your last chance to help me out with this release. If you can, *please* do any final testing with your own Magma programs. Make backup copies of your r39 repositories and then open them directly with r40. You will see a warning and then it should convert them seamlessly. Hilaire said he had a problem with the conversion process, so I would really appreciate, if there is a problem, we can reproduce it together before the release goes out. Thanks, Chris

1 0

r41Alpha1 (was: Re: Counter in Gjaller)
by Chris Muller 01 Sep '07

01 Sep '07

I have just posted my initial solution to this problem to the MagmaTester project known as the configuration "r41Alpha1". After thinking about the problem, it really boils down to you want a syncronized application function. Therefore I have introduced MagmaSession>>#executeInServer: oneArgBlock to run an arbitrary block of code in the server's session, with full access to the persistent model. mySession executeInServer: [ : repositoryController | repositoryController session commit: [ ...increment your counter... ] ] This brings radical new power to Magma, teasing with the idea of an "application server" not just an object server. But there are caveats. For now, it was just the easiest and cleanest way to get what you want; it was so simple I was able to implement it over my lunch hour. Gorgeously, it leverages all existing functionality and integrates into the entire system seamlessly without introducing any new special-cases. But there are caveats. I also included a test case that proves it works across multiple clients in separate images connected to a remote server each grabbing #getNextKey as fast as they can and then verifying they got no duplicates between them and that their union is consecutive and complete. See MagmaTestCase>>#testSequenceNumberGenerator. You may also use it as an example for how you can implement it for your Q2Txn's. However, note you don't have to use a MagmaCounter the way the test case does, if you don't want to. You could simply use an Integer that you commit: [ nextNumber: self nextNumber+1]. Enjoy, but please don't use this radical new power for anything else; I can't promise I will keep this as the solution because.. there are caveats.. On 8/22/07, Göran Krampe <goran(a)krampe.se> wrote: > Hi Chris! > > > Hi GÃ¶ran, no, I'm sorry to hear that since I spent a good 4 days on it > > and ended up quite pleased with the result. > > No need to be sorry! :) I haven't looked but IIRC from your post you > implemented an RC counter a la GemStone, right? Which is quite useful for > counting stuff of course. :) > > > What's the problem with it? > > Well, we have multiple MagmaSessions creating Q2Txn objects, giving them a > strictly increasing unique number and adding them to a MagmaCollection. In > short we want the illusion of a very long OrderedCollection. :) > > In an OrderedCollection each object of course has its position and if you > succeed adding an object to it (without conflicts) then its number == its > position. > > With a MagmaCollection we need to give it a number ourselves (and use an > index for it) - and the problem is how we generate that number so that we > don't end up with multiple Q2Txns with the same number - and not having to > do multiple commits. > > In other words - an "integer series generator" is not exactly the same > problem as a "reduced conflict counter". Note btw that I am describing > this from MY understanding - Balazs and Levente are the ones who are > coding on this and perhaps I have misunderstood some parts :) > > The "simplistic" solution is of course to let all MagmaSessions make a > server call (to the Magma server) which has a counter wrapped up in > critical: blocks for increasing it. Thus making sure we serialize all > "increments". This is what Keith implemented which apparently did not work > in the Magma version I tried a week ago (the server seems to not deal with > the MagmaServerRequest properly when running as a separate image - causing > the client image to "hang" waiting for a response. I logged all Magma > requests going and all seems fine until this "MagmaServerRequest" coming > through the line, haven't debugged it further on the server side - failed > to see where to put the darn halt. :)). > > So.... best would be if you, Keith, Balazs and Levente came up with a > solution. I am ccing them all here (don't have Levente's email handy but > Balazs talks to him). Also - I started installing r40Gamma2 (or was it 3?) > last night but haven't had time to test it in this regard yet. > > regards, Göran > > PS. Martin McClure from GemStone has contacted me and they are interested > in porting Gjallar to GemStone/S! If they go through with that we end up > with a formidable comparison platform for Magma vs GemStone. We will not > leave Magma of course - no worries there - Gjallar is open source and we > also have our offline mechanism which demands an "embeddable" db like > Magma. And in most respects it works really, really good and I hope > Gjallar will become the MAJOR poster app for showing what Magma can do. > > Reply Reply to all Forward Reply Reply to all Reply to allForward Forward Print Add Göran to Contacts list Delete this message Report phishing Show original Message text garbled? Göran Krampe to me, magma show details Aug 24 You're Da Man Chris! :) regards, Göran Reply Reply to all Forward Invite Göran to Gmail Reply Reply to all Reply to allForward Forward Print Add Igor to Contacts list Delete this message Report phishing Show original Message text garbled? Igor Stasenko to goran, me, magma show details Aug 24 I implemented 1:1 as in the sample code in test. And its not working :/ Raising commit error in the middle of Db creation. I don't know what is the cause (the Q2Counter code is fairly simple to make something wrong). Maybe we need some additional changes in gjallar code to conform with latest magma version. Also, if using old code, all works ok - simple gjallar DB created and populated without errors, but once i using executeInServer: it stops with error. I'm not too high expert in magma to suppose what is wrong. I submitted snapshot with new Q2Counter code into gjallar repo, give it a try. On 24/08/07, Göran Krampe <goran(a)krampe.se> wrote: > You're Da Man Chris! :) > > regards, Göran > > _______________________________________________ > Magma mailing list > Magma(a)lists.squeakfoundation.org > http://lists.squeakfoundation.org/mailman/listinfo/magma > -- Best regards, Igor Stasenko AKA sig.

1 0

Re: MagmaCollectionReader>>size takea a long time
by Burella Juan M. 01 Sep '07

01 Sep '07

Thanks Igor for your answer. If calculating a MagmaCollectionReader size is costly, then why not forward #size to its MagmaCollection? My problem is I'm sending #size to a MagmaCollectionReader because I need to paginate the collection for showing its contents in the UI. Any suggestions? Thanks Juan M. On 8/31/07, Igor Stasenko <siguctua(a)gmail.com> wrote: > > use collection. > a #size for MagmaCollectionReader is not something which can be fast, > because collectionreader designed for reading/filtering collections > with provided filter query. It's impossible to determine the number of > items in filtered collection in other way than reading and testing > full collection, thats why #size is bad practice for using with > collectionreader. It designed for sequential reading, not for random > access. > > > On 31/08/2007, Burella Juan M. <juan.burella(a)gmail.com> wrote: > > Hi list, > > > > I have a indexed MagmaCollection ( it has about 11,000 objects). I > > have noticed that requesting the size to a MagmaCollectionReader takes a > > long time whereas the same requesting to a MagmaCollection doesn´t > > > > For example: > > > > "MagmaCollection>>size - fiirst evaluation" > > [ (magmaSession root at: #Serial) size ] timeToRun "answer about 525 > > milliseconds" > > "Note: magmaSession root at: #Serial answer a MagmaCollection" > > > > > > "second evaluation" > > [ (magmaSession root at: #Serial) size ] timeToRun "answer about > > 1milliseconds" > > > > "MagmaCollectionReader>>size first evaluation" > > [ ((magmaSession root at: #Serial) read: #issnString) size ] timeToRun > > "answer about 9798 milliseconds" > > > > "second evaluation" > > [ ((magmaSession root at: #Serial) read: #issnString) size ] timeToRun > > "answer about 8992 milliseconds" > > > > Any suggestions for reducing this time? > > > > Thanks in advance. > > Juan Matias. > > > > _______________________________________________ > > Magma mailing list > > Magma(a)lists.squeakfoundation.org > > http://lists.squeakfoundation.org/mailman/listinfo/magma > > > > > > > -- > Best regards, > Igor Stasenko AKA sig. >

2 2

MagmaCollectionReader>>size takea a long time
by Burella Juan M. 01 Sep '07

01 Sep '07

Hi list, I have a indexed MagmaCollection ( it has about 11,000 objects). I have noticed that requesting the size to a MagmaCollectionReader takes a long time whereas the same requesting to a MagmaCollection doesn´t For example: "MagmaCollection>>size - fiirst evaluation" [ (magmaSession root at: #Serial) size ] timeToRun "answer about 525 milliseconds" "Note: magmaSession root at: #Serial answer a MagmaCollection" "second evaluation" [ (magmaSession root at: #Serial) size ] timeToRun "answer about 1milliseconds" "MagmaCollectionReader>>size first evaluation" [ ((magmaSession root at: #Serial) read: #issnString) size ] timeToRun "answer about 9798 milliseconds" "second evaluation" [ ((magmaSession root at: #Serial) read: #issnString) size ] timeToRun "answer about 8992 milliseconds" Any suggestions for reducing this time? Thanks in advance. Juan Matias.

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Magma September 2007