[etoys-dev] Current project serialization format?

[Andreas Raab]

Yoshiki Ohshima wrote:
> At Tue, 01 Dec 2009 10:59:53 -0800, Andreas Raab wrote:
>> Quick question: What is the current project serialization format used in 
>> Etoys? Historically, image segments were used but I know that Yoshiki 
>> looked at alternatives and I don't know what the current state of 
>> serialization in Etoys is.
> 
>   For regular projects, we still use good old ImageSegments.  The
> alternative is an S-expression representation (dubbed "SISS", homage
> to SIXX) that is used for the QuickGuides contents.
> 
>   The reason for using SISS for QuickGuides is that ImageSegment is
> faster for bigger projects than SISS, but for smaller projects SISS is
> faster and (tends to be) smaller.

But robustness in the face of a changing environment wasn't the main 
driving force? This is a bit where I'm headed - I'm interested in 
updating projects with a storage mechanism that can work robustly in the 
face of a constantly changing environment. I had hoped that SISS might 
be a good starting point for that. BTW, is SISS explicitly key-value 
based or just a sequence of objects with meaning implied by sequence?

>> Specifically I am interested in:
>> - Does an Etoys project file contain any sort of Manifest that states 
>> version dependencies? (i.e., says which version of Etoys it was created 
>> with)
> 
>   Yes. There is a file called manifest in the zip and it looks like:
> 
> Squeak-Version: etoys4.0
> Squeak-LatestUpdate: 2325
> File-Name-Encoding: utf-8
> Project-Language: en
> URI: http://squeakland.org/etoys/yoshiki-3431931444
> user: yoshiki
> Project-Format: ImageSegment
> projectkeywords: fractal, chaos
> projectcategory: 553
> projectname: Mandelbrot
> projectdescription: Mandelbrot set drawing
> projectauthor: 

Ah, very good. This is an excellent start. What I really want to do is 
to provide enough information so that a project loader can tell whether 
it will be able to load a project and if not, why, instead of our old 
friend  "Reading an instance of GobblyGook. Which modern class should it 
translate to?" which is my second-favorite questions to ask users. (of 
course my absolute favorite is when project saving invites you to "stop 
and take a look" at some blocks it encountered - both are completely and 
utterly incomprehensible questions to users that it's pointless to ask 
them in the first place)

In any case, being able to have a common manifest that we can use to 
identify version and possibly other dependencies will be great.

> There was a bit of attempt to provide a higher-level format in SISS.
> It involved like when there is no TransformationMorph, its
> renderedMorph's bounds is in global coordinates, but once
> TransformationMorph is added the it becomes in local.  A higher level
> description should be uniformly dealing with one of them (probably
> local), but it wasn't really to a point where one can save and load a
> project.  For Etoys, any UI object is potentially scriptable and
> reshapable(so far, people may want to restrict it), so it is another
> stumbling block for it.

It's not all that difficult; but my main interest here was really to 
find out what kind of work had already been done for defining serialized 
abstractions for common Object/Morph types (which is admittedly a 
daunting task when you start from scratch) but...

> For a completely different project VPRI are doing internally, I
> store UI objects in SISS in more higher level description of UI
> objects and already went through some major object shaping
> successfully.

... this sounds as if most of that hasn't been done in Etoys.

>> - Assuming it isn't image segment based how are scripts and script 
>> references implemented? Same as previously (file out for scripts, class 
>> vars or globals refs for references) or differently?
> 
>   In the ImageSegment, at least the references are made project local;
> the dictionary of player reference name to actual object is in the
> property of PasteUpMorph and the script compiler looks up objects from
> there.  But otherwise it is similar to before.

How interesting! This is almost as if a project had a namespace 
associated with it, which is an idea that I might like to explore a 
little more.

>   In SISS, a script is saved as a "parse tree" out of send node,
> repeat node, variable node, etc.  It roughly look like:
> 
>   (script :name "script1" 
>      (send :selector "color:sees:"
>          (variable "self)
> 	 (Color "....")
> 	 (Color "....")))
> 
> or such.  The Player class names and internal names of Players are
> abstracted away.

Oh, but if references are looked up via the current project, why is this 
necessary for the players? Shouldn't they resolve unambigously? Also, 
why a parse tree instead of source code? Is there extra information 
that's not available via parsing source? Or is it to avoid a compiler 
dependency?

>> Thanks for any info. I haven't looked at project serialization in many 
>> moons so I'm definitely not up to speed here.
> 
>   Well, I regard that you are still the expert when it comes to
> project saving in various forms....

This is all great info. I definitely have some digging to do in the 
current etoys image ;-)

Cheers,
   - Andreas