<br><br><div class="gmail_quote">On Tue, Nov 2, 2010 at 12:52 PM, Hannes Hirzel <span dir="ltr"><<a href="mailto:hannes.hirzel@gmail.com">hannes.hirzel@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Norberto<br>
<br>
Just thinking aloud ....<br>
76000 records is not that much. Assuming a record is 2kB (2000Bytes)<br>
it would 152000kB or 152MB. Which would be fine for an image. So you<br>
might do it as a memory database -- e.g. Sandstone.<br>
<a href="http://www.squeaksource.com/SandstoneDb" target="_blank">http://www.squeaksource.com/SandstoneDb</a><br>
<br>
See in particular the idea mentioned by Dan Ingalls to go for a really<br>
big image. In fact an image of let's say 400MB would probably not yet<br>
be considered to be big in his terms.<br>
<br>
What do you think?<br></blockquote><div>In theory, that's what everybody should do. I allways dreamed that.<br></div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
Any experience reports with images of this size keeping data?<br></blockquote><div><br>I tried using collection in memory. They consume much more time. Searching in middle sized collections in memory is impossible. <br>
Anyway, the size of the squeak image is a problem I think is not fixed. Image size sometimes grows to much and there's no garbage collecting that can return it to its original size. I had to reconstruct the whole code from the packages several times in order to gain decens of ghost mb. I think Seaside is the problem, but I'm not sure.<br>
I think if you use an original big image, these problem should be dramatic.<br><br>I'll try Sandstone. <br>Thanks for the tip.<br><br></div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
<br>
Regards<br>
<font color="#888888">Hannes<br>
</font><div><div></div><div class="h5"><br>
On 11/2/10, Norberto Manzanos <<a href="mailto:nmanzanos@gmail.com">nmanzanos@gmail.com</a>> wrote:<br>
> The question is "Magma could be used in a real-world application?"<br>
> I think not.<br>
> I used Magma a couple of years ago. I had to migrate 13000 records from a<br>
> database, build objects form the data and find duplications. The whole<br>
> process of migration to Magma took more than two weaks. Yes, weaks!. 15<br>
> days!. And I had to made a lot of strange things to reduce time. I made<br>
> periodical cleaning up process in the middle of the whole migation to<br>
> improve performance. The first attemps tend to infinite, I reduce it to two<br>
> weaks.<br>
> The application was in production two years, with a lot of performance<br>
> problems. Finally we have to throw it away.<br>
> Now I'm trying again. Now I use very simple objects (just a couple of string<br>
> variables) with only one index for one of this strings .<br>
> But the problem is once again the same: each time I add an object to a<br>
> MagmaCollection I have to look after duplicates, and if found, merge them.<br>
> I'm adding object to 4 MagmaCollections, with an index with size 64. (I need<br>
> more, but once again, performance...)<br>
> Look at this numbers<br>
> I tried with 2000 records (total is 76000).<br>
> If I just add the objects without search it takes 2 minutes<br>
> If I search the objects it takes 16 minutes!<br>
> If I search the objects, and then iterate the results to make a more fine<br>
> comparision, it takes 20 minutes!<br>
><br>
> In a linear progression, total time for 76000 records would be 12 hours,<br>
> which is not very good. But time doesn't grow lineary but exponentially.<br>
> I tried with 12000 records: 5 hours 30 minutes. How must I supose it will<br>
> take with 76000 records. 2 days? 3?<br>
> These are not times for a process I surely will perform again as my model<br>
> grows.<br>
> The numbers tells, once again, that the problem is not adding the objects,<br>
> merging or materializing them but searching on a MagmaCollection. Time goes<br>
> by and this problem is not solved.<br>
><br>
> I'm desperate.<br>
> I have no way of persist my objects if I want to avoid everybody laughing at<br>
> me.<br>
><br>
> Norberto<br>
><br>
> On Thu, Oct 21, 2010 at 3:33 AM, Igor Stasenko <<a href="mailto:siguctua@gmail.com">siguctua@gmail.com</a>> wrote:<br>
><br>
>> Hello all,<br>
>> sorry for cross-posting. :)<br>
>><br>
>> I'd like to ask you, if anyone could share either an image or<br>
>> installation with application,<br>
>> which using Magma OODB.<br>
>> I'd like to use it & test how changing different aspects of Magma<br>
>> internals could affect the performance.<br>
>><br>
>> There's many tricks, which is known by Chris how to speed it up by<br>
>> cleverly fine-tuning various Magma options,<br>
>> like read strategy etc.<br>
>> But what i'd like is to see, is some setup, used by people, and by<br>
>> taking it, see how it could make run faster,<br>
>> without changing an application code.<br>
>><br>
>> I remember, someone gave a talk @ ESUG, that they were using Magma for<br>
>> their application,<br>
>> but then forced to switch to another DB layer, because they had bad<br>
>> performance issues.<br>
>> It would be good, if you could give me the code, so i can run it and<br>
>> see if things could be improved.<br>
>> Its not a problem, if code is not open-source, we could sign an NDA,<br>
>> if this is necessary.<br>
>><br>
>> I need something real, simply because benchmarks sometimes not<br>
>> representative. :)<br>
>><br>
>> --<br>
>> Best regards,<br>
>> Igor Stasenko AKA sig.<br>
>><br>
>><br>
><br>
><br>
> --<br>
> Norberto Manzanos<br>
> Instituto de Investigaciones en Humanidades y Ciencias Sociales (IdIHCS)<br>
> FaHCE/UNLP - CONICET<br>
> Calle 48 e/ 6 y 7 s/Nº - 8º piso - oficina 803<br>
> Tel: +54-221-4230125 interno 262<br>
><br>
<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><font face="Arial" size="2">Norberto Manzanos<br>Instituto de Investigaciones en
Humanidades y Ciencias Sociales (IdIHCS)<br>FaHCE/UNLP - CONICET<br>Calle 48 e/
6 y 7 s/Nº - 8º piso - oficina 803<br>Tel: +54-221-4230125 interno
262</font><div style="display: inline;"></div><div style="display: inline;"></div><div style="display: inline;"></div><br>