On Tue, Nov 2, 2010 at 12:52 PM, Hannes Hirzel <hannes.hirzel at gmail.com>wrote:

> Norberto
>
> Just thinking aloud ....
> 76000 records is not that much. Assuming a record is 2kB (2000Bytes)
> it would 152000kB or 152MB. Which would be fine for an image. So you
> might do it as a memory database -- e.g. Sandstone.
> http://www.squeaksource.com/SandstoneDb
>
> See in particular the idea mentioned by Dan Ingalls to go for a really
> big image. In fact an image of let's say 400MB would probably not yet
> be considered to be big in his terms.
>
> What do you think?
>
In theory, that's what everybody should do.  I allways dreamed that.

>
> Any experience reports with images of this size keeping data?
>

I tried  using collection in memory. They consume much more time. Searching
in middle sized collections in memory is impossible.
Anyway, the size of the squeak image is a problem I think is not fixed.
Image size sometimes grows to much and there's no garbage collecting that
can return it to its original size. I had to reconstruct the whole code from
the packages several times in order to gain decens of ghost mb.  I think
Seaside is the problem, but I'm not sure.
I think if you use an original big image, these problem should be dramatic.

I'll try Sandstone.
Thanks for the tip.


>
> Regards
> Hannes
>
> On 11/2/10, Norberto Manzanos <nmanzanos at gmail.com> wrote:
> > The question is "Magma could be used in a real-world application?"
> > I think not.
> > I used Magma a couple of years ago. I had to migrate 13000 records from a
> > database, build objects form the data and find duplications. The whole
> > process of migration to Magma took more than two weaks. Yes, weaks!. 15
> > days!. And I had to made a lot of strange things to reduce time. I made
> > periodical cleaning up process in the middle of the whole migation  to
> > improve performance. The first attemps tend to infinite, I reduce it to
> two
> > weaks.
> > The application was in production two years, with a lot of performance
> > problems. Finally we have to throw it away.
> > Now I'm trying again. Now I use very simple objects (just a couple of
> string
> > variables) with only one index for one of this strings .
> > But the problem is once again the same: each time I add an object to a
> > MagmaCollection I have to look after duplicates, and if found, merge
> them.
> > I'm adding object to 4 MagmaCollections, with an index with size 64. (I
> need
> > more, but once again, performance...)
> > Look at this numbers
> > I tried with 2000 records (total is 76000).
> > If I just add the objects without search it takes 2 minutes
> > If I search the objects it takes 16 minutes!
> > If I search the objects, and then iterate the results to make a more fine
> > comparision, it takes 20 minutes!
> >
> > In a linear progression, total time for 76000 records would be 12 hours,
> > which is not very good. But time doesn't grow lineary but exponentially.
> > I tried with 12000 records: 5 hours 30 minutes. How must I supose it will
> > take with 76000 records. 2 days?  3?
> > These are not times for a process I surely will perform again as my model
> > grows.
> > The numbers tells, once again, that the problem is not adding the
> objects,
> > merging or materializing them but  searching on a MagmaCollection. Time
> goes
> > by and this problem is not solved.
> >
> > I'm desperate.
> > I have no way of persist my objects if I want to avoid everybody laughing
> at
> > me.
> >
> > Norberto
> >
> > On Thu, Oct 21, 2010 at 3:33 AM, Igor Stasenko <siguctua at gmail.com>
> wrote:
> >
> >> Hello all,
> >> sorry for cross-posting. :)
> >>
> >> I'd like to ask you, if anyone could share either an image or
> >> installation with application,
> >> which using Magma OODB.
> >> I'd like to use it & test how changing different aspects of Magma
> >> internals could affect the performance.
> >>
> >> There's many tricks, which is known by Chris how to speed it up by
> >> cleverly fine-tuning various Magma options,
> >> like read strategy etc.
> >> But what i'd like is to see, is some setup, used by people, and by
> >> taking it, see how it could make run faster,
> >> without changing an application code.
> >>
> >> I remember, someone gave a talk @ ESUG, that they were using Magma for
> >> their application,
> >> but then forced to switch to another DB layer, because they had bad
> >> performance issues.
> >> It would be good, if you could give me the code, so i can run it and
> >> see if things could be improved.
> >> Its not a problem, if code is not open-source, we could sign an NDA,
> >> if this is necessary.
> >>
> >> I need something real, simply because benchmarks sometimes not
> >> representative. :)
> >>
> >> --
> >> Best regards,
> >> Igor Stasenko AKA sig.
> >>
> >>
> >
> >
> > --
> > Norberto Manzanos
> > Instituto de Investigaciones en Humanidades y Ciencias Sociales (IdIHCS)
> > FaHCE/UNLP - CONICET
> > Calle 48 e/ 6 y 7 s/Nº - 8º piso - oficina 803
> > Tel: +54-221-4230125 interno 262
> >
>
>


-- 
Norberto Manzanos
Instituto de Investigaciones en Humanidades y Ciencias Sociales (IdIHCS)
FaHCE/UNLP - CONICET
Calle 48 e/ 6 y 7 s/Nº - 8º piso - oficina 803
Tel: +54-221-4230125 interno 262
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20101102/90c6640b/attachment-0001.htm