Trying to understand forwarding proxies

[Norbert Hartl]

Chris,

On 02.02.2011, at 23:58, Chris Muller wrote:

> 
> Cool!  I've really wanted to look at Amazon ECC to see if Magma could
> run there, just haven't had time to do it.
> 
I'm just starting myself looking into it and I'm reading a lot. I was amazed by names like SimpleDB at first. But then I recognized that it is really simple and that using it with more complex models can be even more cumbersome than using an RDBMS. So I'm looking for alternatives.

> Please use the latest 1.2alpha.  I am on the cusp of publishing Magma
> 1.2 right after Squeak 4.2 is all done.
> 
>> If I understand it correctly then
>> 
>> - magma uses a directory to write its files. That could be called the repository
> 
> Yes.
> 
>> - one repository is served by one server at any time
> 
> One repository is served by one or more servers at a time.
> 
Does this mean that magma handles file locking of files and transaction/timestamping data inside the files to be safe to have any number of images accesing it? That would be great. If so could you give information how the locking is achieved/when are commits rejected. Or better how fine/grain coarsed the locking mechanism is.

> 
>> - forwarding proxies can be made from server to server. So these are cross-domain/cross-repository
> 
> Yes, but I would prefer to say, "from repository to repository".  A
> MagmaForwardingProxy is just a "bookmark", or a "soft-link" to another
> object in another repository.  It only persists the 'location' and
> 'oid' of the remote object.  It implements #doesNotUnderstand: to look
> it up and cache it the first time so that subsequent access is fast.
> However, unlike MagmaMutatingProxy's, the ForwardingProxy will never
> become: the remote object.  It will always forward through
> #doesNotUnderstand:, so if you are sending to a FP in a inner-loop,
> send #realObject to the FP to get the cached object for a fast send.
> 
> MagmaForwardingProxies are not intended to be transparent to the Magma
> developer, they must be used deliberately.  MagmaMutatingProxies are
> supposed to be transparent, but there are cases where a #yourself is
> needed (e.g., to avoid the proxy being sent as an argument to a
> primitive).
> 
> The thing to be very aware of about using FP's is that it does tie the
> two repositories together.  The app needs them both running to work.
> But the separation has performance and organizational advantages.
> 
Ok. Seems like a nice feature that opens a whole new bunch of possible use cases. I can imagine that can be very useful if there is a balanced domain partitioning in place. I assume that copying a tree from one repository to another is just a detach/attach operation, right? Forwarding proxies can close some gaps I think.

>> - mimicking a domain model by using forwarding proxies. So objects partitioned over multiple repositories appear to be in the same domain
> 
> Just to be clear, ForwardingProxies have no relation to HA.  HA is for
> replicating one logical repository.  FP's are for linking one logical
> repository (which could be hosted HA) to another logical repository
> (which could also be hosted HA).  This is the configuration I run for
> my own internal app; two repositories, each one HA, so 4 servers total
> (but just two physical machines).
> 
I agree. I didn't mix HA with scaling. Btw. is the master server automatically taken over by another one if the master goes down?

>> - using multiple sessions to read from multiple servers. That would be the case of domain model partitioning
>> 
>> Up to here I would like to know what is the tradeoff in using forwarding proxies. Is the whole communication done via the proxy on the first machine or is another session created to which the client has direct access? Or to be more precise: If I would have a forwarding proxy to a collection in another repository that would hold objects from a third repository and I would detect: an object from that collection to which server am I talking when invoking a method on that detected object?
> 
> A FP only points to one object in one repository.  The FP, itself, is
> a persistent object residing in one repository.  In general, the only
> requirement for the client app to do this is, whereever it SETS the
> object that you want remotely-linked, just send
> #asMagmaForwardingProxy to it.
> 
> When that is committed, the object it refers to MUST be already
> persistent in its own repository so an appropriate location of that
> remote object can be determined and persisted with the FP.
> 
I assume you mean the referred object needs to have been persisted before to create an oid that is used in the proxy?

> Later, when another session comes along and, pretending that FP _is_
> the remote object, sends it a message the proxy itself does not
> understand.  Trace the code starting at
> MagmaForwardingProxy>>#realObject to see that it goes and looks to see
> if a session to that is already present and, it so, uses that one.
> Otherwise, a new session is established.
> 
Is that a normal MagmaSession like I create in my app? If so, can I access automatically created sessions?

Let me bother you with some scenarios. Just to be sure I understood you correct.

Let's take 10 machines (EC2) that have all attached a shared storage all of them could access the same repository in a safe way, right? If this is possible then locking will prevent scaling when a specific number of accessing processes is reached. I could then partition the user data into multiple repositories and still have the ability to access them from multiple machines or to switch responsibility of a repository from one host to another? Furthermore it would be possible to have a central server that keeps information (that is not easy to partition) and that references/is referenced via forwarding proxies from other servers? And mutating proxies might help to lower forwarding requests by materializing the object in another repository. And if it is needed I could even store images that recover automatically when restarted again?

Well, that sounds to good to be true;

thanks,

Norbert


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/magma/attachments/20110203/5340d4c2/attachment-0001.htm