How to do data merging

[Chris Muller]

Magma itself does not have such a data-merge function.  Such a
function would require quite a bit of code and complexity for Magma,
and invariably would not be exactly what different applications would
want.  Applications are therefore responsible for handling this.

However, there are a couple of options to think about:

  - Magma supports long transactions.  Instead of having separate
databases, users could connect to the one master db.  Objects that
were cached in the image can be used and changed while "offline".
Just be sure to save the image before exiting it.  Then, when
network-connection to Magma is back available, restart the image and
commit.  The problem is, you don't have access to all the objects
while offline, just the cached ones.  However, application meta-data
is often not very big, and so a read-only copy of that repository,
accessed via FowardingProxy, could be deployed locally to help
mitigate this.  These meta-data repositories could be easily kept up
to date by the application simply applying log files from the
read-write meta-data repository.

  - Check out Gjaller.  As far as I know, Gjaller solved the exact
problem you are talking about via a rich Command-pattern
implementation.

Cheers.

On Wed, Feb 3, 2010 at 12:36 PM, Dave Woodward
<davewoodward at promedmedical.net> wrote:
> I'm using Magma in a project where there are users who have their own
> offline Magma database for data collection.
>
> When these users have an internet connection, their data will be uploaded
> and "merged" with the data in the Magma database on the server.  Both the
> server and the client are running Pharo.
>
> The user will pretty much be the master of their own subset of data, however
> the data could be changed on the server.
>
> My question is if there is already any functionality within Magma to perform
> some sort of "data merge"?  I think I'll have to make the algorithm myself.
>  Right now I'm thinking of making some sort of a "changeset" on each
> individual object in the database that tracks changed to specific ivars, and
> the merge would compare read/write timestamps and then fall back on
> examining the individual changes, and finally throw some error and allow the
> user to manually choose the winning data.
>
> I've seen, but never used, the MVCC and change conflicts that CouchDB does
> on a per-document basis and I'm kind of thinking in those terms.
>
> But I'd just like to know if there is already something out there that I may
> not know about that I can use.
>
> Any feedback would be appreciated!  Thanks!
>
> --
> Dave Woodward
> ...................................................
> www.promedmedical.net
> davewoodward at promedmedical.net
>
> _______________________________________________
> Magma mailing list
> Magma at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>