Mariano,

The latest version is SRP 3.1 010.0. It was made for GS/S and can be downloaded from the GemStone tech support site. The changes were brought back into VA because ENVY is used to manage source for all environments. Tools from an old ENVY-based VW image are able to load the code and produce the fileins for all other dialects. The new version is compatible with any other 3.x version; however, 3.1 won't be available for most other dialects until I generate the fileins (and parcels). BTW, SRP 3.x adds a couple marshal directives that older versions wouldn't know what to do with if the data contained them; other than that, they are all compatible.

There had been more SRP documentation but most was lost from a hard drive failure many years ago. The class-side documentation methods are the best source. They were updated in 3.1.

#1 - Graph cycles

Cycles are supported by default. You can disable cycle checks for specific objects, types, or entirely if you'd like. Saving uses something like an IdentityDictionary to associate a position; when an object is revisited in a graph, a relative offset is saved instead. Loading uses placeholders. The graph is originally loaded as a graph of placeholders; the placeholders are stripped incrementally as practical or at end of load. SRP uses placeholders because not all dialects have an efficient two-way become:.

#2 - Atomics

Not a problem. If the dialect is even capable of creating a duplicate of those types, the post load action answers the canonical form.

#3 - Set hashing

SRP saves the contents of hashed collections rather than the structure. Objects are added to a new instance of a hashed collection. The loading dialect hashes as it expects to.

#4 - Class reshape

SRP saves Metastate in data (or to separate metastate table) to describe the form that state is to be loaded. SRP can load object state even if the class no longer is defined; in that case an SrpState instance is loaded that has attributes and structure of saved object. Like nearly every other marshaler, new inst vars have nil if saved state has no value. Likewise, a saved attribute value for a removed inst var is ignored by default when loaded. SRP has a wide range of ways to declare mapping and substitution rules. It needs that ability so that kernel objects can be saved in one dialect and loaded into another.

#5 - Saving closures

Not in a portable way. By default, you'll save a non-portable form just like any other object. You may have to define depth rules for saving, and remap rules for loading. If you want a portable form then you could look into ways of saving the source for a block (if you can obtain it for your dialect) and then recompile when loaded. SRP is designed for portability, but some things were just too tricky to make portable between dialects.

SortedCollections are typically the only object that references a block closure. One technique (that I like to see more of) is to have a class-side method that answers a block so that you can compare and reference the block instead of saving it directly. You can add that yourself or do things like create a new sorted collection just to get the default sort block to compare. You can do similar tricks if your domain objects reference blocks.

Hints:

SrpConfiguration (or subclass) should be the starting point for everything you do. Every aspect of SRP is configurable, and there are usually several ways that the same thing can be done.

SRP is fast for what it does, but it was designed with an emphasis on cross-dialect portability first. If you'll never need to save an object in one dialect (like GS/S) to load in another (like VW) then the native marshaling framework may be faster--out of the box. The thing is, since everything is easily configurable in SRP, you can usually make up most performance differences through configuration and tuning.

The fastest marshaling frameworks have objects that encode their data directly in sequence to a stream--and then have trouble loading anything when rules change. Some encode selectors to perform in a loading image (scary). SRP does neither of these. SRP encodes metastate in a simple and compact form, and that specifies what rules are used to load state. Metastate can be stored in a table (similar to an XML DTD file), and a table can be referenced by encoded data. If you do that then you can have performance of direct encoding but without the schema change risks. If you were to compare SRP performance to another marshaler then you should use a metastate table to get a more fair comparison.

SRP has the ability to specify an encoding version; however, the encoding rules have never changed. The data is encoded as a simple sequence of unsigned integers of unlimited size. It is a portable encoding, but also an efficient one. You may be surprised that many double-byte strings save to a size not much larger than a single-byte string. This happens because most the characters of double-byte strings are ASCII characters in a 7 bit range. The high bit is used to as a carry/join flag for the character value.

Floats are a obnoxious objects to make portable. By default, SRP saves the float as the numerator/denominator fractional value. This avoids all float size/structure issues; however, fraction/rational conversion from a float is not precise and may be an approximation when loaded. It is simply how floats/double work. I'd looked into ways to make save/load floats/doubles to the limits of their precision; however, that is difficult to make portable. Some dialects (like GS/S) deny structural access. You need to compute float components (mantissa, exponent, sign) yourself. I'm not happy with how SRP saves floats/doubles. You may need to define your own mapping rules for floats so that they remain as imprecise as their native bit structure for your dialect.

SRP was a fun hobby for me. There are many unique features in SRP. It has the portability of XML, but is less verbose. It would be easy to define a loader that loads XML directly from SRP encoded data. With that, SRP also gains the human-readable features of XML while still being extremely space efficient. SRP was just a hobby though. I have other interests now. SRP is still useful after all these years, but if you want to use it then you'll have to figure out most of it yourself and customize it the way you like.

Paul Baumann


________________________________
From: Mariano Martinez Peck [mailto:marianopeck at gmail.com]
Sent: Thursday, August 19, 2010 9:12 AM
To: Paul Baumann; Pharo Development; The general-purpose Squeak developers list
Subject: Questions about SRP
Importance: High

Hi folks/Paul. First, sorry for the cross-posting. Let me know if I should ask this in another place.

I am interested in SRP and to understand the differences with SmartRefStream, ImageSegment, SIXX, parcels, etc. First I would like to find as much info as possible. Right now I found:

http://techsupport.gemstone.com/entries/175881-srp-3-1-007-0
http://sourceforge.net/projects/srp/
http://map.squeak.org/accountbyid/94ec671f-6615-4ece-9ab0-5d4addc3be8d/package/bae4b8c3-210f-48b6-8819-eae30354d021

These seems to be dead:
http://www.ccse.kfupm.edu.sa/~sohel/IMPP/references/About+State+Replication+Protocol+(SRP).html
http://www.whysmalltalk.com/Smalltalk_Solutions/sts2004/pdf/Baumann.pdf

And....class side methods #documentation

is there something else?

Now, my questions:

1) Does SRP supports cycles in my subgraph? If true, how do you achieve that?

2) What happens with objects like nil, true, false, or any other object that has to be unique in Smalltalk. I mean, how can I make sure that when I load the subgraph in another image, I won't have duplicates of nil, true, etc ?

3) Does it provide a way to fix problems like having to rehash Set instances or things like that?

4) Does it supports class reshape?  suppose I put in the file objects of a certain class and then I load the file in another image where that class has different shape (re-order of instVar, rename, new insVar, etc...). If true, how do you achieve that?

5) Do you handle BlockClosure (or BlockCOntext), MethodCoontext, CompiledMethods and things like that ?

Thanks a lot for any hints you can give me.

Mariano


________________________________
This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20100819/837bd94c/attachment.htm