Image as database (was: Re: Minnow WIKI Migration)

Philippe Marschall philippe.marschall at gmail.com
Tue Oct 24 19:04:26 UTC 2006


2006/10/23, Bert Freudenberg <bert at freudenbergs.de>:
> Am 23.10.2006 um 22:55 schrieb Philippe Marschall:
>
> > 2006/10/23, Bert Freudenberg <bert at freudenbergs.de>:
> >> Am 23.10.2006 um 21:30 schrieb Philippe Marschall:
> >>
> >> > 2006/10/23, Cees de Groot <cdegroot at gmail.com>:
> >> >> On 10/23/06, Philippe Marschall <philippe.marschall at gmail.com>
> >> wrote:
> >> >> > > > So about 300 Euros?
> >> >>
> >> >> [...]
> >> >>
> >> >> > 64bit VM?
> >> >> >
> >> >> You pay the hosting bills for a new box? ;)
> >> >
> >> > I'm willing to pay 2 GB of RAM if that's what is needed to run
> >> Pier.
> >> > That Squeak can't handle this is a Squeak specific limitation
> >> that has
> >> > nothing to do with the point that memory is that cheap.
> >> > As pointed out numerous times on squeak-dev and disputed by
> >> none, all
> >> > VM related issues can be fixed easily by just fixing the VM.
> >> This is
> >> > no problem since the VM is open source.
> >>
> >> If we had a transactional virtual object memory that's continuously
> >> saved to disk (think OOZE/LOOM), that might be viable. Perhaps with
> >> Magma you could have almost the same semantics, just be careful what
> >> you touch. But not with the current object memory. No way. Not if you
> >> care about the data.
> >>
> >> It's not about RAM being cheap or not. It's about designing for the
> >> common and the worst case. Why you would want to bring in gigabytes
> >> of data if the working set is just a few megabytes is beyond me.
> >
> > The point was just that holding the whole wiki in the memory is no
> > problem memory or money wise.
>
> No, this was not the point at all. The point was that *even* if you
> could have as many Gigabytes of RAM as you want, holding everything
> in the image *without* being backed by some permanent storage does
> not scale, and therefore is unsuited for real deployment.

Let me quote Michael Rueger:
> IIRC this does not pull in the history. For the SmallWiki port Thomas
> back then wrote an importer that imports everything. The persistency
> also avoid having to keep everything in memory which with the amount of
> content on Minnow is not practical anyways. I know memory is cheap, but
> not that cheap ;-)

Having no permanent storage (save image doesn't count) is just plain
stupid and therefore is unsuited for real deployment. But this has
nothing to do with holding the whole data in the image. You can save a
page to the filesystem when it was edited or created and still have it
in the image. Pier has hooks for this since before it was called Pier.

Having all the data in RAM scales the same way as having all the data
on disk. Linearly. IIRC Google can hold almost the entire web in RAM.
So there is virtually no limit to that. I know this is not clever. I
just say it is possible and the cost is not excessive (holding Minnow
in RAM, not the web).

> > That the vm, like in many other cases too, is the real problem (and
> > I'm quite sure the Java VM would be up to it) is a completely
> > unrealted issue.
>
> It's news to me that the Java VM supports an object image. Or that
> any real-world system on Java would just load a snapshot of *all* its
> data and save it *in whole* later - I sincerely doubt that.

I was talking about dealing with 2 GB of RAM.

Philippe



More information about the Squeak-dev mailing list