Slow performance

[Chris Muller]

Hi Todd, 300*25 years is 7500 per security..?  If you know you will
always use that entire chunk of data at a time (i.e., for historical
analysis) then you will definitely want to employ a ReadStrategy to
make sure it grabs it in one gulp.  See the documentation for details
about ReadStrategy's.

Otherwise, if you will be accessing a few quotes here, a few quotes
there (incidentally, Magma shines brightest with this type of
application), you need to use a MagmaCollection as Sebastian remarked.
 MagmaCollections allow huge collections to be accessed a page at a
time.

Magma will never be as fast as ImageSegments.  ImageSegments will
never be as flexible as Magma.

Regards,
  Chris

On Nov 5, 2007 2:38 AM, Todd Blanchard <tblanchard at mac.com> wrote:
> I may be doing this wrong so I'll share my approach - I'm finding it
> unusably slow though.
>
> I'm working on a stock market analysis tool.  There are about 1100
> securities on the AMEX, NASDAQ, and NYSE.  So I create a Dictionary
> in the root dict called securities.  This is keyed by stock ticker
> symbol and the value is an object of type security which contains
> fields tickerSymbol, exchange, issueName, and historicalData.
>
> The historicalData is an OrderedCollection of quotes going back as
> far as the 80's.  A quote is a timestamp, hi, low, close, and trade
> volume.  There are around 300 of these per year - going back as much
> as 25 years.  Believe it or not, I can fetch these from yahoo as a
> csv and process them into objects in about 5-10 seconds per
> security.  Saving this data structure into magma seems to take many
> times that.  Something like 3 hours in Magma I think.  If this were
> just the initial load, it would be tolerable.  However fetching the
> last 5 days quotes and splicing them onto the tail of the
> historicalData collection takes as long as the initial load.  So this
> approach isn't working for me.
>
> I'm open to ideas on better ways to structure this.  I'll also be
> adding some charts to keep in the database - the idea being the
> charts are mostly up to date and I only have to replot the last day's
> worth of data everytime it is fetched.  When I say chart - I mean a
> data structure containing a 2D array of values - not a visual
> representation.  I will always draw the visual form on the fly.
>
> This chart will also potentially be a big block of data that will
> reference the historicalData points.
>
> Ideas?  I'm close to just going to image segments - one per security.
>
> -Todd
> _______________________________________________
> Magma mailing list
> Magma at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>