Fwd: Magma

[Chris Muller]

Hi Ramiro,

I hope you don't mind my copying the Magma list since this is a useful
discussion for everyone.

== Scalability ==

In the context of web applications there are two tiers of scaling to
more users.  First, use multiple client sessions across different
images all accessing a server in its own image.  Second, serve the
domain from multiple Magma servers.  Let me explain in order.

-- Tier 1, Multiple clients --

First, you configure to run multiple MagmaClient sessions.  What you
do is have each web session have its own MagmaSession view of the
repository, and use ReadStrategy's and #stubOut: to keep only enough
in memory for the current working web page.  This was proposed years
ago here:

 http://wiki.squeak.org/squeak/5817

and I think Brent may have successfully implemented the multi-image
configuration described on that page under "Remote Connections".
Goran and I have discussed it in the past, but I don't know whether he
ever tried it in earnest.  If so, I hope he will comment about this.

That is the first tier.  Before describing the second tier, let me
first remind that the design of Magma is such that the client does 80%
of the heavy-lifting; a remote server will mostly sit idle, starving
for requests to process until you get around 10 (or more, depending on
lots of factors), busy clients.

Until then, it doesn't do any good to do move to the second tier
(multiple servers) until your clients can swamp the remote server,
causing it to begin to queue the requests it receives from the
multiple client sessions, making them wait.

-- Tier 2, multiple servers --

When you get to this point, you can then begin to break up the server
duties across multiple repositories in several ways.

1) Partition your domain by its major categories and use
MagmaForwardingProxy's to link to objects across repositories.  For
more information:

 http://wiki.squeak.org/squeak/5604

I think this is a smart thing to do from the beginning simply for
organizational purposes, but it also helps performance by dividing up
the server burden across multiple images/computers.

This feature works and has been around for years, although I don't
know anyone who has used it (because no one has ever asked about it).

2) Many domains have information that becomes immutable after it is
created.  For example, transactions in an accounting system, messages
in a communication system, digitized external artifacts, etc.

Making one or more separate "warm backups" provides a mostly
up-to-date copy of your repository that are on-line for, for example,
user queries and reporiting.  This can alleviate considerable burden
from the main repository, which is concerned with the still-dynamic
portion of the persistent model.  For more information:

 http://wiki.squeak.org/squeak/5724

== Faster? ==

Where did you hear the new version is much faster?  Perhaps you are
referring to Sig's new Dictionary's that sped things up?  That may be
the case but I haven't had time to look at them yet.  Thanks to Sig,
Magma now includes the ability to specify whatever
WeakIdentityKeyDictionary you want (see MagmaPreferences), so you can
try out some other Dictionary's but I can't promise I can support them
(or a Magma implementation that uses them).

They need to be tested.  The first step would be to run the entire
test suite with the replacement Dictionary(s).  If those pass then you
can be pretty confident, but I would burn them in in a heavily-used
test environment for a good month first.

Beyond this, Magma r40 is not any faster than 39, except for
single-clause query's, which are 100X faster due to a bug fix.

== 3.10 ==

The official 3.10 is supported, but I have not tried the 3.10
"Development image" yet.  Unless Damien has changed low-level stuff
there shouldn't be any problem.  Again, the way to answer this
question for yourself is to do what I would do to give you the answer;
run the MagmaTester test suite.  Documentation on how to do this is
spelled out on the Magma home page on the Swiki.

 http://wiki.squeak.org/squeak/2661

I'll bet it will be fine, but if you encounter a problem, please let
me know because I do intend to support 3.10.

 - Chris
- Hide quoted text -


On 8/29/07, Ramiro Diaz Trepat <ramirodt at gmail.com> wrote:
>
>
> ---------- Forwarded message ----------
> From: Ramiro Diaz Trepat <ramirodt at gmail.com>
> Date: Aug 28, 2007 6:24 PM
> Subject: Magma
> To: Chris Müller <chris at funkyobjects.org>
>
> Hello Chris,
>    I am happy you are about to release a new Magma.
>    I wanted to ask you a couple of questions.  Is there a way to make
> Magma scale, on a cluster with different servers? In case of course
> one had a lot of queries from a very active web site for instance.  Is
> there a way to have several servers running on the same database files
>  or with replication?
>    Will it run on Damien Cassous' version of Squeak 3.10?
>    I've heared that the new version is much faster, do you have some
> figures?
>    Thanks
>