Stephen Pair wrote:
> Ok, I'm not sure I get all this...if there were three participants in a 
> replicated island, are you saying that if for some reason a message 
> cannot be delivered successfully to one participate (and I'm thinking 
> specifically of the case where that participant is overloaded), that 
> effectively that replica goes into some disconnected state?  Presumably 
> it would later re-sync state with one or more of the other islands once 
> it recovers.  However, I thought Croquet had some kind of a graceful 
> degradation that wouldn't result in a less powerful machine being 
> ejected outright.  (btw, sorry that I'm asking these questions and 
> haven't studied Croquet more closely, I've read the papers and played 
> around with it a little, but haven't dug into the code much)

Not only that, there's also the issue that what we have as application 
policy and what is required by Croquet messaging isn't necessarily the 
same thing ;-)

In this case, the application policy is that we consider any participant 
(avatar) who hasn't been sending his heartbeat within a specified period 
of time to be "in trouble". What the reason for this trouble is doesn't 
matter - whether that's a disconnect or whether it's a computation 
taking too much real time. We indicate this by said exclamation mark but 
that has no bearing on the connection aspect. So a participant in this 
state can indeed catch up and recover and we see this regularly. So what 
you're saying above is basically correct except from the "effectively 
disconnected" bit because whether the client is disconnected or not is 
unknown and only if it's not disconnected it will be able to recover 
"gracefully" (which here means without having to resynchronize its state 
which is always the alternative).

> So...thinking about how Croquet might allow more graceful degradation: 
> does Croquet enforce these strict requirements around successful queuing 
> of messages to all replicas, but the replicated state effectively 
> consists of just the model of the world and not any (or not a lot) of 
> the objects involved in rendering the world?  And, since most of the 
> computational load would be in rendering, Croquet would, in fact, allow 
> for a lot of degradation of the rendering and a participant would not 
> get disconnected just because the rendering can't keep up with 30 fps? 

Yes, that's exactly the case. Rendering is decoupled from message 
processing and basically we're rendering "stable states" inbetween 
messages. Since message processing executes at higher priority, your 
rendering rate will degrade slowly if more time is needed for the 
replicated computation.

> As long as the replicated island (consisting of just an abstract model 
> of the world and not requiring a lot of computational load) is able to 
> keep up with the message flow, the participant would remain connected?

Yes. Although, in the case of our application policy, if there are 
issues in the non-replicated bits that affect the sending of your 
regular client heartbeat we would still treat this as being "in trouble" 
from the application perspective.

> Also, how does Croquet deal with really large worlds?  Do you subdivide 
> a large world into hectares or something (or the 3D equivalent of 
> hectares...is there a word for that?) and do you just replicate the 
> hectare the avatar currently resides in and the immediately adjacent 
> hectares...and make the hectares sufficiently large that you would 
> always have enough to render sufficiently far into the distance?  What 
> about objects that might sit on the boundary of multiple hectares?

Portals. Spaces can be seamlessly linked to each other by portals which 
we usually represent as doors etc. Each portal is also a potentially 
separate authentication domain meaning that you can protect your own 
spaces and only people with the proper permissions will be able to enter 
(replicate and join) them. It does get a bit weird when you see people 
vanishing into doorways that you can't open due to lack of permission 
but such is the strangeness of virtual spaces :-)

Cheers,
   - Andreas