[squeak-dev] Big data, big objects (was: FloatConstants?)

David T. Lewis lewis at mail.msen.com
Thu Dec 18 01:30:00 UTC 2014


On Wed, Dec 17, 2014 at 04:30:10PM -0800, tim Rowledge wrote:
> 
> >> Because I need speed and efficiency more than precision.  I'll be
> >> loading _billions_ of 64-bit Squeak Floats into a indexing system 
> 
> I?m more than a little curious about a system that needs to handle billions of anything; that?s one honkin? great image we could be talking about!
>

It does sound like an interesting problem. Chris seems to be focused on exporting
to an external indexing system, but I think that we should be able to manage data
sets of that scale directly in a Squeak memory. 

The 64-bit Spur object memory, with partitioning of object memory spaces and with
improved garbage collection, is likely to make this a real practical option that
can be implemented on very low cost hardware platforms.

Even now, the simple 68002 object format on an intepreter VM enables huge
object memories on low cost personal computers, so to me the potential of a
more efficient object memory and garbage collector seems clear. I'd like to
try loading some of Chris' billions of Floats into a 68002 memory just to see
how it runs (slowly I presume, but still an interesting experiment - Chris,
can I try it?). I expect that if object memories of that scale work at all
on the old 68002 format, then they are likely to work very well indeed on a
64-bit Spur object memory.

Dave



More information about the Squeak-dev mailing list