[etoys-dev] Current project serialization format?

[Yoshiki Ohshima]

At Tue, 01 Dec 2009 21:26:57 -0800,
Andreas Raab wrote:
> 
> Yoshiki Ohshima wrote:
> > At Tue, 01 Dec 2009 10:59:53 -0800, Andreas Raab wrote:
> >> Quick question: What is the current project serialization format used in 
> >> Etoys? Historically, image segments were used but I know that Yoshiki 
> >> looked at alternatives and I don't know what the current state of 
> >> serialization in Etoys is.
> > 
> >   For regular projects, we still use good old ImageSegments.  The
> > alternative is an S-expression representation (dubbed "SISS", homage
> > to SIXX) that is used for the QuickGuides contents.
> > 
> >   The reason for using SISS for QuickGuides is that ImageSegment is
> > faster for bigger projects than SISS, but for smaller projects SISS is
> > faster and (tends to be) smaller.
> 
> But robustness in the face of a changing environment wasn't the main 
> driving force?

  Sure.  One possibility was to use a version of such things as a
vehicle ot migrate the implementation to somewhere else.  But as you
figured out, no good definition of such robust version of definition
is designed yet.

> This is a bit where I'm headed - I'm interested in 
> updating projects with a storage mechanism that can work robustly in the 
> face of a constantly changing environment. I had hoped that SISS might 
> be a good starting point for that. BTW, is SISS explicitly key-value 
> based or just a sequence of objects with meaning implied by
> sequence?

  I "think" that the sequence of these attributes nor the sequence of
sub expressions doesn't matter, except that the parse tree for a
script.  (But it is not thoroughly tested in that regard).

  The idea is to make it isomorphic to XML, but just happened to
rendered with round parenthesis.

> >> Specifically I am interested in:
> >> - Does an Etoys project file contain any sort of Manifest that states 
> >> version dependencies? (i.e., says which version of Etoys it was created 
> >> with)
> > 
> >   Yes. There is a file called manifest in the zip and it looks like:
> > 
> > Squeak-Version: etoys4.0
> > Squeak-LatestUpdate: 2325
> > File-Name-Encoding: utf-8
> > Project-Language: en
> > URI: http://squeakland.org/etoys/yoshiki-3431931444
> > user: yoshiki
> > Project-Format: ImageSegment
> > projectkeywords: fractal, chaos
> > projectcategory: 553
> > projectname: Mandelbrot
> > projectdescription: Mandelbrot set drawing
> > projectauthor: 
> 
> Ah, very good. This is an excellent start. What I really want to do is 
> to provide enough information so that a project loader can tell whether 
> it will be able to load a project and if not, why, instead of our old 
> friend  "Reading an instance of GobblyGook. Which modern class should it 
> translate to?" which is my second-favorite questions to ask users. (of 
> course my absolute favorite is when project saving invites you to "stop 
> and take a look" at some blocks it encountered - both are completely and 
> utterly incomprehensible questions to users that it's pointless to ask 
> them in the first place)

  Hehe, I agree.

> In any case, being able to have a common manifest that we can uese to 
> identify version and possibly other dependencies will be great.n
> 
> > There was a bit of attempt to provide a higher-level format in SISS.
> > It involved like when there is no TransformationMorph, its
> > renderedMorph's bounds is in global coordinates, but once
> > TransformationMorph is added the it becomes in local.  A higher level
> > description should be uniformly dealing with one of them (probably
> > local), but it wasn't really to a point where one can save and load a
> > project.  For Etoys, any UI object is potentially scriptable and
> > reshapable(so far, people may want to restrict it), so it is another
> > stumbling block for it.
> 
> It's not all that difficult; but my main interest here was really to 
> find out what kind of work had already been done for defining serialized 
> abstractions for common Object/Morph types (which is admittedly a 
> daunting task when you start from scratch) but...
> 
> > For a completely different project VPRI are doing internally, I
> > store UI objects in SISS in more higher level description of UI
> > objects and already went through some major object shaping
> > successfully.
> 
> ... this sounds as if most of that hasn't been done in Etoys.

  That is right.

> >   In SISS, a script is saved as a "parse tree" out of send node,
> > repeat node, variable node, etc.  It roughly look like:
> > 
> >   (script :name "script1" 
> >      (send :selector "color:sees:"
> >          (variable "self)
> > 	 (Color "....")
> > 	 (Color "....")))
> > 
> > or such.  The Player class names and internal names of Players are
> > abstracted away.
> 
> Oh, but if references are looked up via the current project, why is this 
> necessary for the players? Shouldn't they resolve unambigously? Also, 
> why a parse tree instead of source code? Is there extra information 
> that's not available via parsing source? Or is it to avoid a compiler 
> dependency?

  I don't quite get the first question (as I didn't explain it
clearly).  One thing here (I think) is that the Player class name is
still global and say loading the same project twice and then exporting
them would results in different output because of it.  Another is that
when loading a QuickGuide content into a user project, there may be
name collision.

  For using the parse tree, it is partly because to avoid a compiler
dependency (for bringing the code into a different language) and
avoiding parsing overhead.

  From and to the S-expression representation of code, it is possible
to go both ways from tiles and textual code.  So, it is possible to go
from an "arbitrarily edited" (with some side conditions) textual
script to a tile script.  And keep the possibility of switching to
different language, keeping a method as a tree structure makes sense.

> >> Thanks for any info. I haven't looked at project serialization in many 
> >> moons so I'm definitely not up to speed here.
> > 
> >   Well, I regard that you are still the expert when it comes to
> > project saving in various forms....
> 
> This is all great info. I definitely have some digging to do in the 
> current etoys image ;-)

  Don't get disgusted, please ^^;

-- Yoshiki