I am not sure if I understood. In our benchmarks, we have a list of samples and each sample is at the same time an array of objects that we serialize/materialize. For each sample we create instantiate a serializer and a materializer.
I understood now that we should reuse the same serializer/materialize instance for all samples? if true, why I don't use a Singleton ? I mean...it is not clear for me when to instantiate a serializer.
Users of MaObjectSerialization only instantiate one serializer to handle a related "groups" of objects - usually related by being of the same set of classes. In this case, if the samples of your benchmark are small (e.g., < 100 obejcts), you should reuse the same serializer for each sample being serialized/materialized.
A generic pattern for improving real-world performance is to off-load work to an initialization step. For example, many applications pre-cache certain objects from a database at system startup (a.k.a., the "initialization step"). Since startup of the system is only done once, it is ok if it takes, say, an additional 10 or 30 seconds to pre-cache if it means that users will have sub-second response times after the system comes up rather than response times of 5 seconds..
MaObjectSerialization uses this pattern - it is expensive to instantiate a MaObjectSerializer (about 500 milliseconds) but, in exchange, the performance of the serializer is improved.
So if the benchmark is going to include initialization of a new MaObjectSerializer for each of _many tiny_ "samples", then that is not a good measurement of how it would be used in actual practice. The repeated initializations time will dominate 99% of the time consumed, and the "benchmark" would favor the serializers which have fast initialization times but, in fact, may be slower for serialization and/or materialization.
This is why I stressed it is important to measure each operation - initialization, serialization, and materialization - individually, so that interpretation of the results can be made with respect to how it would be used.
It's up to you, of course. But to bring the benchmark into a real-world usage pattern for MaObjectSerializer, I hope you will consider: 1) measure and report initialization, serialization and materialization separately, 2) reuse the same serializer for all of the tiny samples or 3) use very large samples for the benchmark, so that the initialization cost is not such a large factor.
Regards, Chris