Magma May 2006

magma@lists.squeakfoundation.org

5 participants
6 discussions

MagmaTesterLoader-cmm.22
by Chris Muller 31 May '06

31 May '06

While working on the queries implementation, I noticed I'd somehow flipped the logic backward for the performance optimization I did for Bert last month (shouldRemoveGarbageCollectedEntries), negating the performance improvement, sigh. It's really fixed now..! It also includes the new backup method requested by Göran. Enjoy, Chris

1 0

Queries
by goran＠krampe.se 26 May '06

26 May '06

Yo Brent and Chris! I have been toying a bit with Chris latest - but... well, some issues of course. Attached are some silly fixes (I think) but then I had problems related to the fact that indexHashesFor: returns an array of hashes - not all code like that, like for example: mergeAdditionsInto:where: or localSegmentWhere: (the sortblock fails since the key is an array and not a hash) The send of #isEmpty I added in one method was because the segment I got had keys nil and empty objects. Well. :) Another question: how is the reader meant to behave? I mean, if I send (the one I added for convenience) asOrderedCollection to it - then I get all objects. But if I send: reader select: [:each | reader includes: each] then I get the filtered objects. But I assume I shouldn't have to do that? :) regards, Göran

1 0

Backups?
by goran＠krampe.se 25 May '06

25 May '06

Hi Chris and all! Ok, we are getting closer to first test deployment of our app for a few user groups and (since we may experience problems, like db corruption (heaven forbid) or simply bugs in the app causing havoc) I want to make extra sure that I have a good approach regarding backups. Am I correct that since we now have fault tolerance I could essentially just copy the db files to get a backup of the db - even though it will not be transactionally correct? I mean - I can lose the ongoing txns, but the copy should otherwise be able to recover and work, right? Or do I need to make a shutdown in order to make sure I get a "proper" backup? regards, Göran

2 4

rollbacks in magma
by Burella Juan M. 19 May '06

19 May '06

Hello, I newbie in magma. I want to test the begin/rollback on a transaction. I think that the rollback action is made by the abort message. But when I run this test sometime it fail. aMagmaSession := MagmaSession openLocal: 'C:\Squeak 3.8\Squeak3.8-current-win-full\test'. aMagmaSession connectAs: 'test'. aMagmaSession begin. self deny: (aMagmaSession root includesKey: 'xxxXxxx'). aMagmaSession root at: 'xxxXxxx' put: 'hola'. self assert: (aMagmaSession root includesKey: 'xxxXxxx'). aMagmaSession abort. self deny: (aMagmaSession root includesKey: 'xxxXxxx'). "this assertion sometimes fails" aMagmaSession disconnect. Am I doing everything wright? Do I need to make a garbageCollect? Thanks.

3 3

Multiple indices on MagmaCollections?
by goran＠krampe.se 19 May '06

19 May '06

Hi Chris and Brent! Ok, so our project is moving along fine - it will most probably become open source, I intend to prepare a demonstration for OOPSLA and see if it gets acepted (and tell everyone about Seaside/Magma of course). In the app we are building we intend to offer the users advanced filtering of "cases". This boils down to more or less what you have been discussing about indices and Lava (though I am at this point not interested in Lava - but will definitely be later on). So my question is: What is the current status of this work? And where can I get it? :) I have read through your posts on the subject and it looks great to me. regards, Göran

2 4

Re: get well (Actually how to query and sort!)
by Brent Pinkney 19 May '06

19 May '06

Hi Chris, > Hey man, I'm sorry to hear of your injury, I sincerely hope you are ok! Given the brilliance of your post it doesn't sound like there's any permanent damage thank goodness. I am fine, a bit tender but quite fine, thanks. > I do not yet understand your solution but my cursory read it sounds real good.. I need to read it again and dedicate some cycles to understand it better.. In essense I want to allow the Magma server to estimate whether it can create the bitmap and sort order within a given time and signal an exception if it cannot. In this way the client can determine how long is it preperared to wait. I have spent many a while waiting for Oracle to finish a sorted query, so an upperbound is a good enough compromise for me. The sorting solution is also quite simple. Concider the people collection with indices on familyName and age. Currently we will have five indices on this collection: one for the elements, 2 normal indices and 2 census indices. The main index will hold the oids: 0001 -> Brent Pinkney (age = 33) 0002 -> Tom Pinkney (age = 63) 0003 -> Claire Pinkney (age = 31) 0004 -> Man Kanam (age = 29) NB. (familyName = 'Man') 0005 -> Nelson Madiba Mandela (age = 86) Now concider the same objects in the familyName index: they will be stored in sorted order of familyName, viz: 0004, 0005, 0001, 0002, 0003 (ie, Man, Mandela, Brent Pinkney, Tom Pinkney, Claire Pinkney) The age index will also be stored in sorted order, viz 0004, 0003, 0001, 0002, 0005 (ie, Man, Claire Pinkney, Brent Tom Pinkney, Tom Pinkney, Mandela) So if we maintain a new sorting-index where each entry maps and oid to (in this exampl) two integers, being the position of that oid in the index then we have 0001 -> 3, 3 0002 -> 4, 4 0003 -> 5, 2 0004 -> 1, 1 0005 -> 2, 5 So if we have the following query: query := people where: [ :p | (p familyName from: 'Adam' to: 'Zorro') & (p age between: 32 and: 90) ] orderBy: [ p age ]. query pageSize: 2. [ q prepareWithin: 10 seconds ] on: MaTimeout do: [....]. q do: [ :p | .... ] To satisfy this query we have to a) build bitmap and b) walk the bitmap in the correct order. But from #pageSize we know we only have to find the smallest and second smallest age values. We can use the sort-index above to help. Now, (p familyName from: 'Oscar' to: 'Zorro') matches oids: 0001, 0002, 0003, 0004, 0005. But (p age between: 32 and: 80) only matches oids 0001, 0002, 0005. We can estimate the time it will take to built the bitmap as it bound by the number of oids in the tightest cluase (age) and the number of sort cluases (one). So we decide to build up the bimap index using the age clause as it is 'tighter'. For each oid we check the census index for the #familyName index to see whether the oid survives and set each oid's bit in the index accordingly. If the oid survives, we find its sort order in the age index and maintain the smallest two. At the end of this process the bitmap is set and the sort collection is #( 0001->3 0002->4 ) We store this array or sort assacations in the bitmap. Obviously if we fail to build this bitmap and sort order withing the 10 seconds mandated by the client we return an exception. However if all goes well, we return immediately with a segment containing 0001 and 0002 being 'Brent Pinkney' and 'Tom Pinkney'. We also shuttle our posh bitmap back to the client to keep the Magma server simple. Now, if the client does need the second (or subsequent pages) we know that the next oid must occur after entry 4 in the #age index. So we read each oid from the 5th and check whether it is in the bitmap. We continue reading ad checking oids until #pageSize oids have been found. This is the best solution I can think of. It is also not too scary to implement (I think). Please let me know. Brent

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Magma May 2006