[Seaside] Migrations - Object Database

James Foster Smalltalk at JGFoster.net
Tue Apr 29 04:51:58 UTC 2014


Aaron,

Because of GemStone’s architecture, we don’t think of an object as being “hydrated.” That is, they are represented on disk in the same way they are represented in memory. Every object is an instance of some class; the concepts of “previously committed state” and “current class” aren’t really meaningful. The closest idea is whether there are other classes in the ClassHistory collection.

For performance reasons many developers would prefer not to have the VM do additional checks and migrate an object if the behavior of the old version is equivalent to the behavior of the new version. Next, if an object were automatically migrated by two or more concurrent sessions this would create a commit conflict. Finally, since migration can take significant time and other resources, developers prefer to control when those resources are consumed. Depending on the change, one might choose never to migrate. For example, one could add the following to the “old” class to achieve compatibility with the “new” class:

professions
	^profession
		ifNil: [#()]
		ifNotNil: [Array with: profession].

Now there is less to be gained by migrating.

As to your list, 
	(1) GemStone objects know how to migrate themselves (#’migrate’); 
	(2) #’dbVersion’ could be implemented as '^(anObject class classHistory indexOf: anObject class)’;
	(3) & (4) The migrate methods should be passed the prior state (as in GemStone’s #’migrateFrom:instVarMap:’), not available at any time (so we can garbage collect prior states); and
	(5) While a framework could provide auto-migration, the default approach would be slow and any refinement would likely be application-dependent. 

James

On Apr 28, 2014, at 7:24 PM, Aaron Rosenzweig <aaron at chatnbike.com> wrote:

> Thank you James, Diego, and Chris for your thoughts,
> 
> 1) Lazy Initializers
> Thanks Diego, I see how what you say could work for many cases and is generally good practice even without thinking about migrations. But when it’s something more complex, or there has been a lot of code changes over the week and your only pulling “now” then it wouldn’t handle that. Or when you’ve been making a lot of changes over the month and want to push to production it wouldn’t have a straightforward game plan to follow.
> 
> 2) Gemstone documentation
> Thanks James. I got really exited to look into the docs but in the end felt like it was only half of the solution. It does have a notion of “ClassHistory” which every Gemstone class has. That is cool! It also has the notion of the message “migrateFrom:instVarMap:” Where you have access to the old values and can create the new values. That is also very cool! But migrations are not automatic and you cannot chain them together in a predetermined way. “that’s an exercise for the app developer” as what Chris is suggesting. 
> 
> I see value in having a documented way for everyone to upgrade their objects. Sure, app developers could always roll their own methods but if a unified way was available, it would make it easier for all of us and quicker to move between projects in this community. 
> 
> Maybe we can add this to Dali, so it is available for Magma, GOODS, and Gemstone. Actually any object db that has a layer built into Dali. Here’s a rough idea of what I’m thinking:
> 
> 1) The object knows
> The object should know how to migrate itself. When it is “hydrated” from the data store it compares previously “committed” state to what the current class is like and automatically provides migration after migration to become current. Even if you have not pulled new code from the repo for over a month, it will upgrade your object database for you. You’ll get all the latest and greatest features from your colleagues without any effort. You don’t have to think. 
> 
> 2) #dbVersion
> This method would return a value of 1 as the default. Then every time you make changes to the class that would require a migration you would increment the value. Simply adding a new method that can be satisfied with lazy initializers wouldn’t require a change to #dbVersion. If you were to change #profession -> #professions (go from one to many) then this would require you to increment #dbVersion. 
> 
> 3) #migrate2, #migrate3, … #migrateN
> To go from #dbVersion (1) to (2) then you would define a method called #migrate2. Likewise, to go from #dbVersion (2) to (3) you would define a #migrate3 method. This would be a running history of “how” you migrate to the new hotness of your app.
> 
> 4) #priorState
> Inside each of our #migrateN methods we could reference a #priorState and ask it messages like #profession so that we can build our current ivars. 
> 
> 5) extending Dali
> We could then extend Dali to automatically take our #dbVersion and #migrateN methods to auto-run migrations on our behalf. We could have the choice of doing it as objects are loaded or we could run a command to do them all in a batch like Chris had with his Magma specific command. 
> 
> This idea of full “migrations” doesn’t appear to exist even in a particular commercial flavor such as Gemstone… but maybe through an improved Dali we could all benefit from this approach. Once and for all! No more reinventing the wheel. We all have the same structured game plan no matter what OODBMS we use.
> 
> Sound good?
> AARON ROSENZWEIG / Chat 'n Bike
> e:  aaron at chatnbike.com  t:  (301) 956-2319 		
> 	
> 
> On Apr 28, 2014, at 2:14 PM, James Foster <Smalltalk at JGFoster.net> wrote:
> 
>> Aaron,
>> 
>> As you say, this isn’t really a GOODS-specific question but a more generic one.
>> 
>> The Programming Guide for GemStone/S 64 Bit has a an entire chapter devoted to the topic (see http://gemtalksystems.com/index.php/community/gss-support/documentation/gs64/). 
>> 
>> There are a variety of approaches to changing the schema of an object. First, since Smalltalk has dynamic typing, you don’t have to have a Profession instance in the profession instance variable of Person. You can store either a Profession instance or a Collection of Profession instances. You can then have accessors that return either one or a collection. Alternatively, in GemStone you can add a method to the old class version to return a collection, and create a new version of the class that stores a collection. Then, at your convenience you can migrate all the instances of the old version to instances of the new version.
>> 
>> James
>> 
>> On Apr 26, 2014, at 9:55 AM, Aaron Rosenzweig <aaron at chatnbike.com> wrote:
>> 
>>> Hi,
>>> 
>>> We’ve been experimenting with GOODS as an object database persistency mechanism for a couple of weeks. We plan to use Dali with it too. We’re really excited about this but have a burning question.
>>> 
>>> Question: 
>>> How do we handle changes to the “model” layer? 
>>> 
>>> Example: 
>>> Suppose that last week we had a “Person” and “Profession” object. Each “Person” could only have one “Profession.” We’ve been saving some data to our Object Database and have 500 people and 25 professions currently saved and used with our app.
>>> 
>>> This week we realize that our world view is wrong. Some people are both a “Chef” and a “Computer Scientist.” They have 2 or more professions. We must change our “model.” The class “Person” must change. 
>>> 
>>> How do we fix our object database now?
>>> 
>>> Background:
>>> This isn’t really a GOODS specific question is it? I imagine it is the same issue with Magma, GemStone, etc. But in all the examples I’ve found nobody seems to talk about “migrations” or maybe I’ve overlooked it. Can someone point me in the right direction?
>>> 
>>> In NeXT / Apple WebObjects I would write a “migration” to do the necessary steps to change SQL tables, make new DB constraints, etc. as an executable script. I would commit this migration to the repository. Any of the other developers on our team would automatically get it when they pulled from the repository. They don’t need to think about it. As soon as the WebObjects app launches the migrations would be run and they could see the new functionality and make “Marcus” both a “Chef and a Computer Scientist." When we deploy the new code in production, the migration will automatically run there too. This is all explained in detail here:
>>> 
>>> https://github.com/wocommunity/wonder/blob/master/Frameworks/Core/ERExtensions/Sources/er/extensions/migration/package.html
>>> 
>>> Any helpful pointers are appreciated.
>>> 
>>> Thank you,
>>> AARON ROSENZWEIG / Chat 'n Bike
>>> e:  aaron at chatnbike.com  t:  (301) 956-2319 		
>>> 	
>>> 
>>> _______________________________________________
>>> seaside mailing list
>>> seaside at lists.squeakfoundation.org
>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>> 
>> _______________________________________________
>> seaside mailing list
>> seaside at lists.squeakfoundation.org
>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
> 
> _______________________________________________
> seaside mailing list
> seaside at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/seaside/attachments/20140428/e360184d/attachment.htm


More information about the seaside mailing list