[Seaside] ESUG SummerTalk - Fuel, binary object serializer

[Mariano Martinez Peck]

On Wed, May 25, 2011 at 2:18 PM, Sebastian Sastre <
sebastian at flowingconcept.com> wrote:

> Sounds awesome!
>
> The streaming feature is appealing.
>
>
Yes. At the beginning we were using MultiByteFileStream directly. Then we
use a ByteArray new writeStream as a full buffer of the full graphs.
We used a lot of memory but speed was very good. But it was also not good
because the array was growing all the time.
So, Sven suggested a ZnBufferedWriteStream he was using for Zinc. That way,
we have a buffered write stream were we can set a buffer, say 5000 elements,
and the speed is almost the same as using a full baffer, because ok, we go a
little more to disk but on the other hand the collection buffer doesn't need
to grow.

So....in conclusion, when using a buffered stream writer instead of
MultiByteFileStream  we increased speed about 12x and with a little buffer
of 5000 elements.

I might take a look at it. Won't hurt as a redundant repo backup tactic
>
>
:)


> sebastian <http://twitter.com/#%21/sebastianconcpt>
>
> o/
>
>
>
> On May 24, 2011, at 5:39 PM, Martin Dias wrote:
>
> Hi folks. I am really happy to announce that ESUG is sponsoring me for Fuel
> development through the ESUG SummerTalk. I am Martin Dias, a student at
> Buenos Aires, Argentina. The idea behind this SummerTalk is to implement
> Fuel, a binary, fast and general-purpose object graph serializer in Pharo<http://www.pharo-project.org/>.
> It is based on VisualWorks' Parcels ideas.
>
> Actually, the project has already started since several months. Tristan
> Bourgois and I started with the project while doing an internship with RMoD,
> INRIA <http://rmod.lille.inria.fr/web/pier>. Since a couple of months, Mariano
> Martinez Peck <http://marianopeck.wordpress.com/> joined the team, and now
> he is the official mentor in the SummerTalk.
>
> ESUG website for SummertTalk:
> http://www.esug.org/wiki/pier/Promotion/SummerTalk/SummerTalk2011
>
> The website with all the necessary information is here:
> http://rmod.lille.inria.fr/web/pier/software/Fuel
> It even includes slides explaining the algorithm. In addition, a paper is
> in progress.
>
> For the moment, Fuel already provides the following features:
>
> - Fast pickle format. It is much faster to materialize than to serialize.
> - Correctly support class reshape (when the class of serialized objects has
> changed).
> - Serialize ANY kind of object. For the moment there is no object to our
> knowledge that we cannot serialize and materialize.
> - Be able to completely serialize classes and traits (not just a global
> name).
> - Support cycles and avoid duplicates in the graph.
> - Integration to Moose <http://www.moosetechnology.org/> with an extension
> to export and import their models.
> - Detection of globals: for example if you serialize Transcript, it is not
> duplicated and instead managed as a global reference.
> - Solve common problems like Set rehash.
> - Buffered writing: we use a buffered write stream for the serialization
> part (thanks Sven!).
> - No need of special support from the VM.
> - Try to have a good object oriented design.
> - Well tested (about 120 tests, for the moment).
> - Large set of benchmarks (even benchmarks for Moose extension).
>
> And of course, there are a lot features for the future. You can see some of
> them in the website and some in the issue tracker:
> http://code.google.com/p/fuel/issues/list
>
> We really appreciate all kind of feedback and comments. If you want to try
> it, check in the website how to do it. It is extremely easy.
>
> Once again, I want to thank a lot to ESUG for sponsoring the project. I
> plan to create a "news" section in the website with some RSS. I will keep
> you informed.
>
> Best regards,
> Martin
>
> _______________________________________________
> seaside mailing list
> seaside at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
>
>
> _______________________________________________
> seaside mailing list
> seaside at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
>


-- 
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/seaside/attachments/20110525/cfe16fb0/attachment.htm