[squeak-dev] Looking for real-world Magma application

Hannes Hirzel hannes.hirzel at gmail.com
Tue Nov 2 15:52:53 UTC 2010


Norberto

Just thinking aloud ....
76000 records is not that much. Assuming a record is 2kB (2000Bytes)
it would 152000kB or 152MB. Which would be fine for an image. So you
might do it as a memory database -- e.g. Sandstone.
http://www.squeaksource.com/SandstoneDb

See in particular the idea mentioned by Dan Ingalls to go for a really
big image. In fact an image of let's say 400MB would probably not yet
be considered to be big in his terms.

What do you think?

Any experience reports with images of this size keeping data?


Regards
Hannes

On 11/2/10, Norberto Manzanos <nmanzanos at gmail.com> wrote:
> The question is "Magma could be used in a real-world application?"
> I think not.
> I used Magma a couple of years ago. I had to migrate 13000 records from a
> database, build objects form the data and find duplications. The whole
> process of migration to Magma took more than two weaks. Yes, weaks!. 15
> days!. And I had to made a lot of strange things to reduce time. I made
> periodical cleaning up process in the middle of the whole migation  to
> improve performance. The first attemps tend to infinite, I reduce it to two
> weaks.
> The application was in production two years, with a lot of performance
> problems. Finally we have to throw it away.
> Now I'm trying again. Now I use very simple objects (just a couple of string
> variables) with only one index for one of this strings .
> But the problem is once again the same: each time I add an object to a
> MagmaCollection I have to look after duplicates, and if found, merge them.
> I'm adding object to 4 MagmaCollections, with an index with size 64. (I need
> more, but once again, performance...)
> Look at this numbers
> I tried with 2000 records (total is 76000).
> If I just add the objects without search it takes 2 minutes
> If I search the objects it takes 16 minutes!
> If I search the objects, and then iterate the results to make a more fine
> comparision, it takes 20 minutes!
>
> In a linear progression, total time for 76000 records would be 12 hours,
> which is not very good. But time doesn't grow lineary but exponentially.
> I tried with 12000 records: 5 hours 30 minutes. How must I supose it will
> take with 76000 records. 2 days?  3?
> These are not times for a process I surely will perform again as my model
> grows.
> The numbers tells, once again, that the problem is not adding the objects,
> merging or materializing them but  searching on a MagmaCollection. Time goes
> by and this problem is not solved.
>
> I'm desperate.
> I have no way of persist my objects if I want to avoid everybody laughing at
> me.
>
> Norberto
>
> On Thu, Oct 21, 2010 at 3:33 AM, Igor Stasenko <siguctua at gmail.com> wrote:
>
>> Hello all,
>> sorry for cross-posting. :)
>>
>> I'd like to ask you, if anyone could share either an image or
>> installation with application,
>> which using Magma OODB.
>> I'd like to use it & test how changing different aspects of Magma
>> internals could affect the performance.
>>
>> There's many tricks, which is known by Chris how to speed it up by
>> cleverly fine-tuning various Magma options,
>> like read strategy etc.
>> But what i'd like is to see, is some setup, used by people, and by
>> taking it, see how it could make run faster,
>> without changing an application code.
>>
>> I remember, someone gave a talk @ ESUG, that they were using Magma for
>> their application,
>> but then forced to switch to another DB layer, because they had bad
>> performance issues.
>> It would be good, if you could give me the code, so i can run it and
>> see if things could be improved.
>> Its not a problem, if code is not open-source, we could sign an NDA,
>> if this is necessary.
>>
>> I need something real, simply because benchmarks sometimes not
>> representative. :)
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>>
>
>
> --
> Norberto Manzanos
> Instituto de Investigaciones en Humanidades y Ciencias Sociales (IdIHCS)
> FaHCE/UNLP - CONICET
> Calle 48 e/ 6 y 7 s/Nº - 8º piso - oficina 803
> Tel: +54-221-4230125 interno 262
>



More information about the Squeak-dev mailing list