Magma March 2007

magma@lists.squeakfoundation.org

7 participants
11 discussions

Re: Searching magma string indices
by Chris Muller 04 Apr '07

04 Apr '07

Hi Brent! I think you've cleverly solved your own question.. By indexing the reversed value, its no different than a keyword index where you can already query on the presence of multiple keywords, as in: myArticles where: [ : each | (each keywords equals: 'Squeak') & (each keywords equals: 'Magma') ] So you would just have to convert the wildcards to multiple conjunctions, for example, 'foo%bar' would just have to be translated to: self people where: [ : each | (each familyName startsWith: 'foo') & (each familyName endsWith: 'bar') ] and the index would have something like: startsWith: aString self equals: aString endsWith: aString self equals: aString reversed You don't even need a separate "reversed" index, just add two entries (the forward and reversed) for each object added to the collection. Now, the expression '%foo%bar%' goes beyond the first requirement of just searching on a prefix and/or a suffix.. For this a different type of index would be needed that adds every possible value for "Fredfoon Tobart", the following values: fredfoon tobart redfoon tobart edfoon tobart dfoon tobart foon tobart oon tobart on tobart n tobart tobart tobart obart bart art rt t Your '%foo%bar%' expression would then need to translate to (familyName from: 'foo' to: 'foo' maAlphabeticalNext) & (familyName from: 'bar' to: 'bar' maAlphabeticalNext) to find the object(s) that had "Fredfoon Tobart". ... The standard indexes included with Magma don't begin to scratch the surface of their real potential. I find it fascinating to to talk about this stuff, thanks! It's great to "see" you again buddy, welcome back, Chris ----- Original Message ---- From: Brent Pinkney <brent(a)zamail.co.za> To: magma(a)lists.squeakfoundation.org Sent: Thursday, March 1, 2007 12:15:07 AM Subject: Searching magma string indices Hi, I have been thinking about how one could search Magma String indices for a given prefix or suffix. i.e. find all words starting with 'foo' or ending with 'bar'. One can add SQL-ish "like" functionality for a prefix seach quite easily with the method: MaClause >> #like: aString "aString is a SQL-ish term and may include a trailing %" self from: aString upTo: (aString copyWithout: $%) maAlphabeticalNext ! For example: find all the people whose name starts with 'Jo': self people where: [ :p | p familyName like: 'Jo%' ]. This executes as a very efficient indixed search and will be debuting (in some form) in the next release of Lava. However, finding all the strings ending with 'bar' is more troublesome as there is no way to map this to a #from:to:# expression on the Magma index. UNLESS of course, we maintain another, hidden, String index on the collection with the hash of the string reversed. That is self people where: [ :p | p familyName like '%Jo' ]. becomes: self people read: #familyName_reversed from: 'oJ' to: ... It is very late here, so I have not been able to try it out, but I welcome comments. What would be really nice is an index on expressions of the form '%foo%bar%'. Somehow I doubt this. Cheers Brent _______________________________________________ Magma mailing list Magma(a)lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma

5 9

Latest Lava SQL for Magma OODB
by Brent Pinkney 02 Apr '07

02 Apr '07

Hi, I have just released the latest version of the Lava SQL engine for Chris's Magma OODB. This version is distinguished by being able to optimise some queries to use Magma's indexes instead of materialising each element and checking it in the image. You will need an image with: 1. Squeak 3.9 2. MagmaServerLoader-cmm.25 from Squeaksource 3. Lava-brp.36 from Squeaksource 4. LavaTesting-brp.34 from Squeaksource 5. Open TestRunner and run LavaTestCase and LavaOptimisedTestCase. 88 tests should pass. This creates a Magma repository of poeple and their vehicles in ~/lava. It may take some time to create the LavaTestResource. 6. Open the Transcript and a Workspace. 7. Evaluate: LavaTestResource current schema explore. and explore how lava maps Magma collections in the repository to a relational schema. 8. To see the 'raw' Magma collections, evaluate: LavaTestResource current vehicles explore. LavaTestResource current people explore. 9. Use Magma's complex queries to find and print all the Pink* people who are not too old: (LavaTestResource current people where: [ :p | (p familyName match: 'Pink*') & (p age < 50) ]) do: [ :p | Transcript show: p fullName; tab; show: p age; tab; show: p vehicle; cr ] 10. Evalute this to help see how Lava turns SQL into Smalltalk code: LavaTestResource current session preferences compileProcessorSilently: false; removeProcessorAfterExecution: false; tryToOptimiseQueries: true. 11. Now try a the same query using Lava instead of Magma: (LavaTestResource current session execute: ' select p.fullName, p.age from people p where p.familyName like ''Pink%'' and p.age < 50 ') do: [ :row | Transcript show: row fullName; tab; show: row age; tab; cr ] 12. Browse class LaProcessorLava. This is the class Lava created to execute the SQL. The #executeSubquery1 method contains the logic. Notice how Lava was able to optimise the (p.familyName like 'Pink%') term using a Magma where: clause. 12. Try a join to get the people with vehicles: (LavaTestResource current session execute: ' select p.fullName, p.age, v.registration from people p, vehicles v where p.familyName like ''Pink%'' and p.age < 50 and p.vehicleKey = v.vehicleKey ') do: [ :row | Transcript show: row fullName; tab; show: row age; tab; show: row registration; cr ] 13. Browse LaProcessorLava >> #executeSubquery1 and notice 1. how the join is implemented as a nested loop. 2. that Lava cannot (yet) optimise a join - the (p.familyName like 'Pink%') term is not longer optimised by Magma. 14. To explore the importance of begin able to optimise Lava queries to use Magma's underlying indices, use the LavaLargeTestResource. This Lava test case creates a Magma database of electricity reading transactions. The number of transactions stored may be very large: Evalute: LavaLargeTestResource current createTransactions: 5000. "This may take some time." and LavaLargeTestResource current session preferences compileProcessorSilently: false; removeProcessorAfterExecution: false; tryToOptimiseQueries: true. 15. Now evaluate this query: (LavaLargeTestResource current session execute: ' select * from transactions where deviceId = 222 and agentName = ''midrak'' and reading > 3000 ') do: [ :row | Transcript show: row deviceId; tab; show: row agentName; tab; show: row reading; cr ] This should execute in a few seconds. 16. Browse LaProcessorLava >> #executeSubquery1 and notice 1. how Lava has optimised the (deviceId = 222) and (agentName = ''midrak'') terms 2. The unindexed (reading > 3000) term is not optimised. 17. Re-evaluate with tryToOptimiseQueries: false.in 13) above and retry the query in 15) above. It will take a lot longer. Comments welcomed Brent

2 1

Re: Chris Muller has a new e-mail address
by Chris Muller 27 Mar '07

27 Mar '07

Hi Marcelo, here are the packages, in load-order. Ma exception handling Ma base additions Collections-BTree Ma special collections Ma special collections tester Ma traverse object graphs Ma proxy support Ma object serialization Ma object serialization tester Ma time objects Ma client server Ma client server tester WriteBarrier Magma client Collections-BTree Magma server Ma Armored Code Magma tester Note you can simply install the current release (Magma tester) from SqueakMap you will have most of the above. Here is some quick information about each package: Ma exception handling - Generic superclasses for various kinds of errors (MaUserError, MaSecurityViolation, MaSoftwareError, etc.). Ma base additions - Generic superclass MaObject. Note you may ignore the MaOverrider hierarchy and MaProgressBarMorph, those are used by Maui only (a UI framework for Morphic). Ma special collections - Includes MaHashIndex and MaHashIndexRecord, which are the back-ends for MagmaCollections (large collections) and their indexes. Note you don't need to worry about MaTree, MaPermutator, or any of the MaDictionary hierarchy. The first two are not used by Magma, and MaDictionary only exists to fix a Squeak-specific performance issue I doubt exists in any other Smalltalk. You also don't need WeakValueBTree. Ma special collections tester - SUnit test cases for the above. Ma traverse object graphs - extensions to Object and others for visiting each node in its graph. Ma proxy support - Lightweight mutating and also a forwarding proxy. Inherit from ProtoObject. Ma object serialization - Serialization framework that offers first-class object API to serialized object graphs (e.g., #graphDo: [ : each...). This is essential for Magma because no domain classes are on the server. It is also heavily used by Ma client server. Note 48-bit oids. Ma object serialization tester - test cases for the above. Ma time objects - There may be Dolphin equivalents for these.. Ma client server - Client-server framework uses an easy object protocol. Accesses Squeaks TCP Sockets. See http://minnow.cc.gatech.edu/squeak/2978. Ma client server tester - test cases for the above. Multi-image test requires a special way to invoke (not through SUnit Test Runner). WriteBarrier - Package from Avi Bryant. Although Magma supports it, it defaults to off. If this is not easy to port then don't bother with it, but you might still keep the "writeBarrier ifTrue:[]" logic, just for code consistency with Squeak version.. Magma client - The client package. Uses everything above. Magma server - The server package. Contains a running session for the server session, so requires every package abot. Ma Armored Code - SUnit enhancement for testing network programs in multiple running images (i.e., separate OS threads, real TCP sockets). Magma tester - the full test-suite for Magma. Big and kinda complex, lots of tests. Here are some thoughts you may like to consider for the port. - strive for cohesive interoperability between the Squeak and Dolphin versions. - strive for total file and network compatibility. - to facilitate forward evolution, the code and package structure should try to stay as consistent as possible. - hope a maintainer will be able to apply fix and enhancement deltas to the Dolphin version, test and release. - Do you plan to release with a similar license? (Squeak Magma is MIT) Regards, Chris On 3/26/07, Marcelo Cortez <smalltalker.marcelo(a)gmail.com> wrote: > Hi Cris > > I'm working in port magma-client to Dolphin dialect, montincello > repository has too many packages I be confused, which packages are > server or client? > any clue ,vice > are welcomed > > best regards > MDC > > pd: order of package load are important i think. and testing packages too. > > > > > On 3/22/07, Chris Muller <asqueaker(a)gmail.com> wrote: > > My old e-mail address has been having problems, my new address is: > > > > ma.chris.m (at) gmail.com > > > > Please send all future e-mail correspondence to this address. > > > > If you've sent something to my old address in the last month or so, I may > > not have received it. If you are obliged, I would be happy to read it at my new > > address. > > > > Thank you. > > > > - Chris Muller > > _______________________________________________ > > Magma mailing list > > Magma(a)lists.squeakfoundation.org > > http://lists.squeakfoundation.org/mailman/listinfo/magma > > >

1 0

Chris Muller has a new e-mail address
by Chris Muller 23 Mar '07

23 Mar '07

My old e-mail address has been having problems, my new address is: ma.chris.m (at) gmail.com Please send all future e-mail correspondence to this address. If you've sent something to my old address in the last month or so, I may not have received it. If you are obliged, I would be happy to read it at my new address. Thank you. - Chris Muller

1 0

Re: use of includesAnyOf:
by Chris Muller 14 Mar '07

14 Mar '07

Yes, index the same object at multiple endDates and then "equals" will inherently be "includesAnyOf:".. ----- Original Message ---- From: Hilaire Fernandes <hilaire2006(a)laposte.net> To: magma(a)lists.squeakfoundation.org Sent: Sunday, March 11, 2007 6:41:01 AM Subject: Re: use of includesAnyOf: Hilaire Fernandes a écrit : > I am wondering about the use of includesAnyOf: query. Is it the same use > than inclidesAnyOf: from the collection classes? > > Is the following correct ? > > myMagmaCollection where: [:each | each endDate includesAnyOf: {Date > today . Date yesterday} > > With endDate returning a Date instance. Or may be with endDate returning a collection of Date will be even more useful? Hilaire _______________________________________________ Magma mailing list Magma(a)lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/magma

3 3

Exemple of Magma use
by Hilaire Fernandes 14 Mar '07

14 Mar '07

Hum, I have been occupying this channel quite a bit recently. So I though it may be fair I share a bit what can be done with Magma+Squeak. It is about a networked education oriented application, hooked to the Internet. Learners' learning are modeled through notions network. A standard Morphic application is used to design this network, this is picture (IFICompetencyDrawable). Magma database is used in back end to save all the model of the network. Learner activities (see one example with IFIPedagoBooklet) are going through Morphic application, then each learner act is saved in a Magma Database. Given the oriented object nature of Magma, you can image what kind of data act can be easily stored in the DB. With a rough estimation, I expect, in real situation, the database to grow to several hundred of MB, if not GB Thanks Chris to make Magma. Now I will look at the design of the teacher console with Magma+Seaside part, expect even more question ;-) Hilaire

1 0

Exemple of Magma use
by Hilaire Fernandes 13 Mar '07

13 Mar '07

1 0

use of includesAnyOf:
by Hilaire Fernandes 11 Mar '07

11 Mar '07

I am wondering about the use of includesAnyOf: query. Is it the same use than inclidesAnyOf: from the collection classes? Is the following correct ? myMagmaCollection where: [:each | each endDate includesAnyOf: {Date today . Date yesterday} With endDate returning a Date instance. Hilaire

1 2

Some thought about OODB design
by Hilaire Fernandes 09 Mar '07

09 Mar '07

With SQL, tables are just separated collections, co-related with index key. When using Magma, I found it is pretty elegant to nest the collections. So with Magma we are closer to the inner nature of the designed domain and index key are not much needed. But when it comes to query it is another story. If you need to formulate a querry based on attribute down in the nested collection, it starts to be a bit complicated and probably very expensive. With flat, separated collection as in Sql it is simple. Of course it is possible with Magma to just do the same and to use separated collections, co-related with index key. I am curious to know which options you took for your own design... Hilaire

1 0

Error on server side
by Hilaire Fernandes 09 Mar '07

09 Mar '07

When populating a MagmaCollection I got error with index, because I used the wrong selector for the index. I fix that, then I start to get "class definition unknow" error message, then my whole data is unaccessible (lost), then from the server I stop and restrat (even restarting Squeak), and now each time I start the Magma server I got the two following messages (capture here). I am sure I do a lot of mistake when using Magma, but now I start to get anxious with it. Hilaire

2 9

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Magma March 2007