<div>Thanks you Chris,</div><div>good to know that there is a real world magma application with 24 GB of size.</div><div><br></div><div>See you,</div><div>Facu</div><br><div class="gmail_quote">On Fri, Nov 26, 2010 at 3:43 PM, Chris Muller <span dir="ltr"><<a href="mailto:asqueaker@gmail.com">asqueaker@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">It's a proprietary data-aggregation tool that permutes every<br>
combination of data-attributes of an input-source at multiple levels.<br>
Even small inputs into the system can lead to very large output<br>
graphs. It's a signature test-application for Magma; building and<br>
accessing a fairly large and complex data-structure and something I<br>
think would be very difficult to do, at least abstractly, with an<br>
RDBMS..<br>
<div><div></div><div class="h5"><br>
On Thu, Nov 25, 2010 at 6:56 AM, Facundo Vozzi <<a href="mailto:facundov79@gmail.com">facundov79@gmail.com</a>> wrote:<br>
> Hi Chris,<br>
> what kind of application is that? At work I'm working on a ten years old<br>
> production system with Oracle DBMS and it's 1 GB of size.<br>
><br>
> Thanks for share your analysis again,<br>
> Facu<br>
> On Wed, Nov 24, 2010 at 2:38 PM, Chris Muller <<a href="mailto:ma.chris.m@gmail.com">ma.chris.m@gmail.com</a>> wrote:<br>
>><br>
>> There has been interest in Magma's read and query performance lately,<br>
>> so I thought I would share results of a recent benchmark test.<br>
>><br>
>> It's an actual application which does very heavy reading and writing<br>
>> to a Magma repository. This test was 24GB of repository work, over<br>
>> two days. My goal was to determine, once the persistent model becomes<br>
>> many times the size of RAM and HD access appeared to become the<br>
>> limiting factor:<br>
>><br>
>> - how fast is Magma, in terms of "objects per second" and in<br>
>> "kilobytes per second"?<br>
>> - how fast is this relative to the speed when the repository was empty?<br>
>><br>
>> Conclusion:<br>
>><br>
>> - Magma started at 4K objects per second (empty repository), 282K per<br>
>> second.<br>
>> - Finished with 2.3K objects per second (6GB repository), 750K per<br>
>> second.<br>
>> - Memory consumption by the image never exceeded 300MB.<br>
>><br>
>> It is important to note, these times are from the client MagmaSession<br>
>> point-of-view, including the full server roundtrip plus<br>
>> materialization. Also, as can be seen from the attached data, there<br>
>> were many requests which only brought back one or two objects which,<br>
>> while dramatically lowering the overall reported throughput, is<br>
>> actually a real-world scenario for applications.<br>
>><br>
>> Verbose description:<br>
>><br>
>> When the test first started the repository was tiny, and there were<br>
>> only ~4K server requests during a 5-minute monitored period. The<br>
>> total number of objects read across all ~4K requests was ~17M for an<br>
>> average throughput of 4K objects per second (ops). As the model grew<br>
>> in size and complexity, more and more objects during the 5-minute<br>
>> monitored intervals were required to perform application-posting of<br>
>> the same number of input records; doubling to ~8K server requests<br>
>> during a 5-minute period, however only a few more objects brought back<br>
>> (total 21M), for an average of 2500 ops. 32 hours after that, the<br>
>> rate of reads dropped to about 2300 ops. This is due to two factors:<br>
>><br>
>> - the objects were larger (e.g., more pointers to other objects)<br>
>> - they were less clustered (having been replaced with objects which<br>
>> could not be co-located with the original object)<br>
>><br>
>><br>
>><br>
>> The fact that objects got bigger (e.g., more pointers to other<br>
>> objects) is corroborated by another stat, "the number of *bytes* per<br>
>> second" read off the HD in order to access those objects. Initially,<br>
>> when the test first started, there were about 4K *requests* for<br>
>> objects in one of the first five-minute periods, which were read at a<br>
>> rate of 282K per second. 24 hours later, there were only 2 times as<br>
>> many requests for objects, but were read-and-materialized at a rate of<br>
>> 750K bytes / second.<br>
>><br>
>><br>
>> - Chris<br>
>><br>
>> _______________________________________________<br>
>> Magma mailing list<br>
>> <a href="mailto:Magma@lists.squeakfoundation.org">Magma@lists.squeakfoundation.org</a><br>
>> <a href="http://lists.squeakfoundation.org/mailman/listinfo/magma" target="_blank">http://lists.squeakfoundation.org/mailman/listinfo/magma</a><br>
>><br>
><br>
><br>
> _______________________________________________<br>
> Magma mailing list<br>
> <a href="mailto:Magma@lists.squeakfoundation.org">Magma@lists.squeakfoundation.org</a><br>
> <a href="http://lists.squeakfoundation.org/mailman/listinfo/magma" target="_blank">http://lists.squeakfoundation.org/mailman/listinfo/magma</a><br>
><br>
><br>
</div></div></blockquote></div><br>