[Seaside] Migrations - Object Database

Sat Apr 26 21:13:12 UTC 2014

I use Magma, but yes, Data-quality management applies to any persistent
model, and is a little more involved than at first it seems, because it's
tempting to just spank-out a workspace script which enumerates appropriate
objects in your DB and fixes/upgrades them.

But doing only that is loaded with caveats.  After running a repair script,
can one _really_ feel comfortable the model is "all fixed" and ready for
production?  Did the script really work?  Did it catch all cases or was
there possibly a bug in it?

To address these concerns, Magma employs a first-class DataRepair which is
used to perform data repairs/migrations on Magma models in a controlled and
reliable fashion.  It is based on the repair process we used at Florida
Power and Light for patching/upgrading the models in its GemStone databases.

The requirements of a data-repair are:

- Enumerate all objects needing repair.
- Of the enumerated objects, #check, #count or #identify the ones that are
in need of repair.
- Of the enumerated objects, #improve or #repair the ones that are in need
of repair.
- Before committing #repair, _verify_ whether the repaired objects are, in
fact, repaired by using the same check as for #check, #count and #identify.
 If they are, commit, if not, abort.
- Connect a brand-new session, run a final #check and report whether the
repair was successful or failed.
- (Optional) If successful, you might wish to persist the DataRepair object
itself somewhere in your database, so it has a history of what was done to
it.
- Output report of for each of the above actions.

Here is an example of Magma's DataRepair constructor:

(MagmaDataRepair
at: (MagmaRemoteLocation host: 'prod1' port: 51199)
enumerate: [ : session : repair | session root accountsDo: [ : eachAccount
| repair check: eachAccount ] ]
check: [ : eachAccount : repair | eachAccount importFilters anySatisfy: [ :
each | each class = MaxImportFilter ] ])
repair: [ : eachAccount : repair | eachAccount importFilters withIndexDo: [
: eachFilter : index | eachAccount importFilters at: index put: eachFilter
asKeywordFilter ] ]

To this object, I can send any of the 5 actions:  #check, #count,
#identify, #improve or #repair.  The first three are read-only operations.
 The 4th will commit the repairs as-it-goes, the last one only commits
after a successful pre-verification.

Although you the user must specify the enumerate:, check: and repair:
blocks, the DataRepair object encapsulates the _process_ of repairing
models in a controlled and reliable fashion, by using those user-specified,
model-specific blocks.

 - Chris

On Sat, Apr 26, 2014 at 11:55 AM, Aaron Rosenzweig <aaron at chatnbike.com>wrote:

> Hi,
>
> We’ve been experimenting with GOODS as an object database persistency
> mechanism for a couple of weeks. We plan to use Dali with it too. We’re
> really excited about this but have a burning question.
>
> *Question: *
> How do we handle changes to the “model” layer?
>
> *Example: *
> Suppose that last week we had a “Person” and “Profession” object. Each
> “Person” could only have one “Profession.” We’ve been saving some data to
> our Object Database and have 500 people and 25 professions currently saved
> and used with our app.
>
> This week we realize that our world view is wrong. Some people are both a
> “Chef” and a “Computer Scientist.” They have 2 or more professions. We must
> change our “model.” The class “Person” must change.
>
> How do we fix our object database now?
>
> *Background:*
> This isn’t really a GOODS specific question is it? I imagine it is the
> same issue with Magma, GemStone, etc. But in all the examples I’ve found
> nobody seems to talk about “migrations” or maybe I’ve overlooked it. Can
> someone point me in the right direction?
>
> In NeXT / Apple WebObjects I would write a “migration” to do the necessary
> steps to change SQL tables, make new DB constraints, etc. as an executable
> script. I would commit this migration to the repository. Any of the other
> developers on our team would automatically get it when they pulled from the
> repository. They don’t need to think about it. As soon as the WebObjects
> app launches the migrations would be run and they could see the new
> functionality and make “Marcus” both a “Chef and a Computer Scientist."
> When we deploy the new code in production, the migration will automatically
> run there too. This is all explained in detail here:
>
>
> https://github.com/wocommunity/wonder/blob/master/Frameworks/Core/ERExtensions/Sources/er/extensions/migration/package.html
>
> Any helpful pointers are appreciated.
>
> Thank you,
>   *Aaron Rosenzweig* / Chat 'n Bike <http://www.chatnbike.com>
> *e:*  aaron at chatnbike.com  *t:*  (301) 956-2319   [image: Chat 'n Bike]  [image:
> Chat 'n Bike]
>
>
> _______________________________________________
> seaside mailing list
> seaside at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/seaside/attachments/20140426/12c9fc60/attachment.htm