64 bit VM?

David T. Lewis lewis at mail.msen.com
Sun Apr 23 19:58:22 UTC 2006


On Sun, Apr 23, 2006 at 12:25:43PM -0700, Michael Latta wrote:
> What is the status on the 64 bit VM effort?  Is that VM ready for use?  I am
> considering a project that Squeak would be good for, but I need a very large
> image (20GB-2TB).  Can the garbage collector deal with this?  Is there
> enough locality for the OS virtual memory system to work?  The 64 bit Sun VM
> has been tested with 100s of GB of heap.  Has anything like this been done
> in Squeak?

The short answer would be "no, it's not ready". But it certainly could be made
ready given some motivation and a bit of work.

There was some recent mention of this topic on the vm-dev list. I don't know
if this is archived anywhere, so I'm attaching a copy of the messages.

I don't know how a large object memory will work with respect to garbage
collection etc, but I don't think that anybody has really tried it, so I
would not be inclined to bet on it for a real project just yet. And certainly
I would not expect an image of the size you are describing to be feasible
at this point.

On the other hand, this is a chicken and egg problem. In the absence of
a real project, nobody is likely to really work on 64 bit images and VMs.
So given that this is *your* project, and there are no adverse consequences
to *me* if it does not work out, I say go for it ;)

Dave 

-------------- next part --------------
>From SRS0=V9O0=6M=discuss.squeakfoundation.org=vm-dev-return-627-lewis=mail.msen.com at shell.msen.com  Fri Apr 21 20:15:49 2006
Return-Path: <SRS0=V9O0=6M=discuss.squeakfoundation.org=vm-dev-return-627-lewis=mail.msen.com at shell.msen.com>
Received: from shell.msen.com (shell.msen.com [148.59.86.2])
	by conch.msen.com (8.13.6/8.13.6) with ESMTP id k3M0FmbI026347
	for <lewis at conch-old.msen.com>; Fri, 21 Apr 2006 20:15:48 -0400 (EDT)
X-Sent-To: <lewis at mail.msen.com>
Received: from box2.squeakfoundation.org (box2.squeakfoundation.org [85.10.195.197])
	by shell.msen.com (8.13.6/8.13.6) with SMTP id k3M0FkXW096299
	for <lewis at mail.msen.com>; Fri, 21 Apr 2006 20:15:48 -0400 (EDT)
	(envelope-from vm-dev-return-627-lewis=mail.msen.com at discuss.squeakfoundation.org)
Received: (qmail 5907 invoked by uid 1001); 21 Apr 2006 23:49:06 +0000
Mailing-List: contact vm-dev-help at discuss.squeakfoundation.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Post: <mailto:vm-dev at discuss.squeakfoundation.org>
List-Help: <mailto:vm-dev-help at discuss.squeakfoundation.org>
List-Unsubscribe: <mailto:vm-dev-unsubscribe at discuss.squeakfoundation.org>
List-Subscribe: <mailto:vm-dev-subscribe at discuss.squeakfoundation.org>
Delivered-To: mailing list vm-dev at discuss.squeakfoundation.org
Received: (qmail 5901 invoked from network); 21 Apr 2006 23:49:06 +0000
X-POP-User: tim.rowledge.org
Mime-Version: 1.0 (Apple Message framework v749.3)
Content-Transfer-Encoding: 7bit
Message-Id: <F7E86872-D204-4672-9270-9AF77774C254 at rowledge.org>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
To: squeak vm <vm-dev at discuss.squeakfoundation.org>
From: tim Rowledge <tim at rowledge.org>
Subject: 64 bit cleanup completion?
Date: Fri, 21 Apr 2006 16:49:01 -0700
X-Mailer: Apple Mail (2.749.3)
Received-SPF: None; receiver=shell.msen.com; client-ip=85.10.195.197; envelope-from=<vm-dev-return-627-lewis=mail.msen.com at discuss.squeakfoundation.org>
Received-SPF: None; receiver=shell.msen.com; client-ip=85.10.195.197; helo=box2.squeakfoundation.org
X-Scanned-By: milter-7bit/0.11.137 (shell.msen.com [148.59.86.2]); Fri, 21 Apr 2006 20:15:48 -0400
X-milter-7bit-Pass: YES
X-Spamilter-SPF: pass (policy result: [pass] from rule [a/24]) reciever=shell.msen.com; client-ip=85.10.195.197; envelope-from=<vm-dev-return-627-lewis=mail.msen.com at discuss.squeakfoundation.org>; helo=box2.squeakfoundation.org;)
X-Milter: Spamilter (Reciever: shell.msen.com; Sender-ip: 85.10.195.197; Sender-helo: box2.squeakfoundation.org;)
Status: RO
Content-Length: 739
Lines: 17

It's been a while since the 64 bit cleanup work was started and  
almost as long since anything much was done with it. There are a  
number of plugins that need some work to get rid of things like  
cCoerce:to: in favour of oopForPointer: etc.

There are places with #flag: Dan lying around and Dan has expressed a  
lack of time to do anything with them but a willingness to sit with  
someone and a beer to explain what he was thinking of. I'm not going  
to be heading down to silicon valley anytime soon so I certainly  
can't do that but anyone that can take notes could do it and report  
back. Volunteers?

tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
"Bother" said Pooh, as he flunked the the sobriety test.


>From SRS0=W12J=6M=discuss.squeakfoundation.org=vm-dev-return-628-lewis=mail.msen.com at shell.msen.com  Fri Apr 21 22:37:26 2006
Return-Path: <SRS0=W12J=6M=discuss.squeakfoundation.org=vm-dev-return-628-lewis=mail.msen.com at shell.msen.com>
Received: from shell.msen.com (shell.msen.com [148.59.86.2])
	by conch.msen.com (8.13.6/8.13.6) with ESMTP id k3M2bQGg000159
	for <lewis at conch-old.msen.com>; Fri, 21 Apr 2006 22:37:26 -0400 (EDT)
X-Sent-To: <lewis at mail.msen.com>
Received: from box2.squeakfoundation.org (box2.squeakfoundation.org [85.10.195.197])
	by shell.msen.com (8.13.6/8.13.6) with SMTP id k3M2bOX3001234
	for <lewis at mail.msen.com>; Fri, 21 Apr 2006 22:37:25 -0400 (EDT)
	(envelope-from vm-dev-return-628-lewis=mail.msen.com at discuss.squeakfoundation.org)
Received: (qmail 17705 invoked by uid 1001); 22 Apr 2006 02:10:43 +0000
Mailing-List: contact vm-dev-help at discuss.squeakfoundation.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Post: <mailto:vm-dev at discuss.squeakfoundation.org>
List-Help: <mailto:vm-dev-help at discuss.squeakfoundation.org>
List-Unsubscribe: <mailto:vm-dev-unsubscribe at discuss.squeakfoundation.org>
List-Subscribe: <mailto:vm-dev-subscribe at discuss.squeakfoundation.org>
Delivered-To: mailing list vm-dev at discuss.squeakfoundation.org
Received: (qmail 17699 invoked from network); 22 Apr 2006 02:10:42 +0000
Date: Fri, 21 Apr 2006 22:10:40 -0400
From: "David T. Lewis" <lewis at mail.msen.com>
To: tim Rowledge <tim at rowledge.org>, vm-dev at discuss.squeakfoundation.org
Subject: Re: 64 bit cleanup completion?
Message-ID: <20060421221040.B27160 at conch.msen.com>
References: <F7E86872-D204-4672-9270-9AF77774C254 at rowledge.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <F7E86872-D204-4672-9270-9AF77774C254 at rowledge.org>; from tim at rowledge.org on Fri, Apr 21, 2006 at 04:49:01PM -0700
Received-SPF: None; receiver=shell.msen.com; client-ip=85.10.195.197; envelope-from=<vm-dev-return-628-lewis=mail.msen.com at discuss.squeakfoundation.org>
Received-SPF: None; receiver=shell.msen.com; client-ip=85.10.195.197; helo=box2.squeakfoundation.org
X-Scanned-By: milter-7bit/0.11.137 (shell.msen.com [148.59.86.2]); Fri, 21 Apr 2006 22:37:26 -0400
X-milter-7bit-Pass: YES
X-Spamilter-SPF: pass (policy result: [pass] from rule [a/24]) reciever=shell.msen.com; client-ip=85.10.195.197; envelope-from=<vm-dev-return-628-lewis=mail.msen.com at discuss.squeakfoundation.org>; helo=box2.squeakfoundation.org;)
X-Milter: Spamilter (Reciever: shell.msen.com; Sender-ip: 85.10.195.197; Sender-helo: box2.squeakfoundation.org;)
Status: O
Content-Length: 1639
Lines: 34

On Fri, Apr 21, 2006 at 04:49:01PM -0700, tim Rowledge wrote:
> It's been a while since the 64 bit cleanup work was started and  
> almost as long since anything much was done with it. There are a  
> number of plugins that need some work to get rid of things like  
> cCoerce:to: in favour of oopForPointer: etc.
> 
> There are places with #flag: Dan lying around and Dan has expressed a  
> lack of time to do anything with them but a willingness to sit with  
> someone and a beer to explain what he was thinking of. I'm not going  
> to be heading down to silicon valley anytime soon so I certainly  
> can't do that but anyone that can take notes could do it and report  
> back. Volunteers?

All we need here is a bit of coordination. Maybe just a swiki page.
Way back when, Ian posted a list of "who seems to own what plugin"
and status with respect to 64 bit cleanliness. This is a perfectly
good starting point if we can just put it on a swiki page and update
it from there.

Also needed is a "64 bit Squeak" category for bug tracking on
Mantis. There are image *and* VM issues that will need to be
resolved, but these are of no interest to most folks with 32 bit
Squeak image/VM, and should be separately tracked if anything
is going to be done about them.

The original 64-bit motivators (Dan and Ian) did the hard bit. Now
all we need is to put a little bit of structure around the follow
up tasks. Yes, I will volunteer to pick up one or two tasks, and
I'm sure that other folks will do so also.

Dave

p.s. I think this power of two business is a bit over-hyped. The
correct machine word size is and always has been 24 bits.

>From SRS0=578h=6M=discuss.squeakfoundation.org=vm-dev-return-629-lewis=mail.msen.com at shell.msen.com  Sat Apr 22 00:06:12 2006
Return-Path: <SRS0=578h=6M=discuss.squeakfoundation.org=vm-dev-return-629-lewis=mail.msen.com at shell.msen.com>
Received: from shell.msen.com (shell.msen.com [148.59.86.2])
	by conch.msen.com (8.13.6/8.13.6) with ESMTP id k3M46CtE002391
	for <lewis at conch-old.msen.com>; Sat, 22 Apr 2006 00:06:12 -0400 (EDT)
X-Sent-To: <lewis at mail.msen.com>
Received: from box2.squeakfoundation.org (box2.squeakfoundation.org [85.10.195.197])
	by shell.msen.com (8.13.6/8.13.6) with SMTP id k3M46AtK004092
	for <lewis at mail.msen.com>; Sat, 22 Apr 2006 00:06:11 -0400 (EDT)
	(envelope-from vm-dev-return-629-lewis=mail.msen.com at discuss.squeakfoundation.org)
Received: (qmail 23616 invoked by uid 1001); 22 Apr 2006 03:39:30 +0000
Mailing-List: contact vm-dev-help at discuss.squeakfoundation.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Post: <mailto:vm-dev at discuss.squeakfoundation.org>
List-Help: <mailto:vm-dev-help at discuss.squeakfoundation.org>
List-Unsubscribe: <mailto:vm-dev-unsubscribe at discuss.squeakfoundation.org>
List-Subscribe: <mailto:vm-dev-subscribe at discuss.squeakfoundation.org>
Delivered-To: mailing list vm-dev at discuss.squeakfoundation.org
Received: (qmail 23606 invoked from network); 22 Apr 2006 03:39:29 +0000
In-Reply-To: <20060421221040.B27160 at conch.msen.com>
References: <F7E86872-D204-4672-9270-9AF77774C254 at rowledge.org> <20060421221040.B27160 at conch.msen.com>
Mime-Version: 1.0 (Apple Message framework v746.2)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <BE46ECD7-27A0-46B7-B6E2-E653609FDBF7 at speakeasy.net>
Cc: tim Rowledge <tim at rowledge.org>, vm-dev at discuss.squeakfoundation.org
Content-Transfer-Encoding: 7bit
From: Ian Piumarta <piumarta at speakeasy.net>
Subject: Re: 64 bit cleanup completion?
Date: Fri, 21 Apr 2006 20:39:25 -0700
To: "David T. Lewis" <lewis at mail.msen.com>
X-Mailer: Apple Mail (2.746.2)
Received-SPF: None; receiver=shell.msen.com; client-ip=85.10.195.197; envelope-from=<vm-dev-return-629-lewis=mail.msen.com at discuss.squeakfoundation.org>
Received-SPF: None; receiver=shell.msen.com; client-ip=85.10.195.197; helo=box2.squeakfoundation.org
X-Scanned-By: milter-7bit/0.11.137 (shell.msen.com [148.59.86.2]); Sat, 22 Apr 2006 00:06:12 -0400
X-milter-7bit-Pass: YES
X-Spamilter-SPF: pass (policy result: [pass] from rule [a/24]) reciever=shell.msen.com; client-ip=85.10.195.197; envelope-from=<vm-dev-return-629-lewis=mail.msen.com at discuss.squeakfoundation.org>; helo=box2.squeakfoundation.org;)
X-Milter: Spamilter (Reciever: shell.msen.com; Sender-ip: 85.10.195.197; Sender-helo: box2.squeakfoundation.org;)
Status: RO
Content-Length: 4414
Lines: 89

David,

> p.s. I think this power of two business is a bit over-hyped. The
> correct machine word size is and always has been 24 bits.

24?  24???!!!  What kind of number is that?!?  You can't even fit an  
entire file name into it (sixbit uppercase ASCII, naturally)!

Noooo....  You need 36 bits, my man.  Split down the middle for those  
ordinary everyday tasks where 18 bit halfwords will do just fine, and  
in only half the resources too.

A PDP-10 with a KL10 core driving 40 Newbury terminals.  That's what  
Squeak really needs!

(Seriously: AFAIK DEC never finished the design and implementation of  
the PDP-2 which would have added a 24-bit machine to the most elegant  
range of CPUs ever designed, IMO.  That left either a few 24-bit  
offerings from Honeywell and the like, or move up to glorious 36-bit  
PDP-/DECsystem-10s.)

>> There are places with #flag: Dan lying around and Dan has expressed a
>> lack of time to do anything with them but a willingness to sit with
>> someone and a beer to explain what he was thinking of. I'm not going
>> to be heading down to silicon valley anytime soon so I certainly
>> can't do that but anyone that can take notes could do it and report
>> back. Volunteers?

I could maybe do that next week.  But here's what I remember as the  
last thoughts on the matter:

Plugins:  This is mostly going to be tidying up declarations making  
sure int means machine width, foo * means machine pointer, sqOop  
means Squeak width pointer and sqInt means integer with same width as  
sqOop.  Gratuitous casts between pointer and int all have to be  
replaced with appropriate macros/inline functions.  (In fact I think  
all declarations should honestly and truthfully declare precisely the  
type of their value, including function pointers, with _no_  
exceptions.  While we're at it, functions returning nothing should be  
'void', not 'int'.)  Beyond that is hunting down and repairing 32-bit  
assumptions: divide/multiply by 4, shift by 2, mask by [~]3, etc.   
LargeIntegerPlugin might have some unique surprises waiting there.   
The parts of BitBlt and B3D that were never tested (NOTE: these  
plugins are NOT entirely converted yet) should be tested and repaired  
where needed.  In some cases it's necessary to truncate to 32 bits to  
avoid unwanted carry out of 32 bits or to force sign extension out of  
bit 31.  IIRC, graphics code had a few very hard to find issues along  
those lines.  This is mostly mindless tedium with the occasional need  
to think seriously about the right type to use, made even more  
tedious by having to regenerate the plugin from the image ever few  
minutes having correlated a C line number with a statement in a  
Squeak method somewhere.

Taking advantage of 64-bit 'long's:  We also discussed the two other  
obvious changes: SmallIntegers being represented as 63-bits + tag in  
64-bit images, and 64-bit Bitmaps storing two 32-bit pixels in each  
word -- with a couple of additional primitives for 64-bit accesses.   
It could be the case that many of the 'flag:'s in the image are  
related to those two issues.  Both of these go deep, and profoundly  
affect things both in the Interpreter and in various plugins.  Not to  
be undertaken lightly.

I know Dan also wanted to clean up the base and method headers, in  
particular the split fields, in an incompatible image format change.   
Some 'flag:'s might be attached to those too.

If there was anything else then I'm afraid I can't remember it  
offhand now.

I think I remember Andreas working on the win32 support code to make  
it 64-bit clean(er), but I'm not certain.

Frankly I think we'd be making huge progress if we can just get the  
rest of the plugins to compile and run in both 64- and 32-bits, with  
sufficient confidence in their correctness to fold them back into the  
trunk.

I'd be more than happy to set up a repository for 64-bit support code  
while it's unstable.  I could also, in case of dire need, offer  
accounts on 64-bit hardware for anyone who is eager to work on a  
particular plugin but has no 64-bit hardware to test their changes.

FWIW: one carrot to dangle in front of people as a motivator is that  
this work is only half concerned with running 64-bit images.  The  
other half is about running 32-bit images on 64-bit hardware, which  
more and more people are finding to be a frustrating limitation.

Cheers,
Ian

>From SRS0=w5K0=6M=discuss.squeakfoundation.org=vm-dev-return-630-lewis=mail.msen.com at shell.msen.com  Sat Apr 22 01:51:48 2006
Return-Path: <SRS0=w5K0=6M=discuss.squeakfoundation.org=vm-dev-return-630-lewis=mail.msen.com at shell.msen.com>
Received: from shell.msen.com (shell.msen.com [148.59.86.2])
	by conch.msen.com (8.13.6/8.13.6) with ESMTP id k3M5pmTT005698
	for <lewis at conch-old.msen.com>; Sat, 22 Apr 2006 01:51:48 -0400 (EDT)
X-Sent-To: <lewis at mail.msen.com>
Received: from box2.squeakfoundation.org (box2.squeakfoundation.org [85.10.195.197])
	by shell.msen.com (8.13.6/8.13.6) with SMTP id k3M5pkEd007503
	for <lewis at mail.msen.com>; Sat, 22 Apr 2006 01:51:47 -0400 (EDT)
	(envelope-from vm-dev-return-630-lewis=mail.msen.com at discuss.squeakfoundation.org)
Received: (qmail 29836 invoked by uid 1001); 22 Apr 2006 05:25:05 +0000
Mailing-List: contact vm-dev-help at discuss.squeakfoundation.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Post: <mailto:vm-dev at discuss.squeakfoundation.org>
List-Help: <mailto:vm-dev-help at discuss.squeakfoundation.org>
List-Unsubscribe: <mailto:vm-dev-unsubscribe at discuss.squeakfoundation.org>
List-Subscribe: <mailto:vm-dev-subscribe at discuss.squeakfoundation.org>
Delivered-To: mailing list vm-dev at discuss.squeakfoundation.org
Received: (qmail 29830 invoked from network); 22 Apr 2006 05:25:05 +0000
X-POP-User: tim.rowledge.org
In-Reply-To: <BE46ECD7-27A0-46B7-B6E2-E653609FDBF7 at speakeasy.net>
References: <F7E86872-D204-4672-9270-9AF77774C254 at rowledge.org> <20060421221040.B27160 at conch.msen.com> <BE46ECD7-27A0-46B7-B6E2-E653609FDBF7 at speakeasy.net>
Mime-Version: 1.0 (Apple Message framework v749.3)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <D0248491-3BCD-4815-829A-F7DBA2093800 at rowledge.org>
Cc: "David T. Lewis" <lewis at mail.msen.com>,
        vm-dev at discuss.squeakfoundation.org
Content-Transfer-Encoding: 7bit
From: tim Rowledge <tim at rowledge.org>
Subject: Re: 64 bit cleanup completion?
Date: Fri, 21 Apr 2006 22:24:58 -0700
To: Ian Piumarta <piumarta at speakeasy.net>
X-Mailer: Apple Mail (2.749.3)
Received-SPF: None; receiver=shell.msen.com; client-ip=85.10.195.197; envelope-from=<vm-dev-return-630-lewis=mail.msen.com at discuss.squeakfoundation.org>
Received-SPF: None; receiver=shell.msen.com; client-ip=85.10.195.197; helo=box2.squeakfoundation.org
X-Scanned-By: milter-7bit/0.11.137 (shell.msen.com [148.59.86.2]); Sat, 22 Apr 2006 01:51:48 -0400
X-milter-7bit-Pass: YES
X-Spamilter-SPF: pass (policy result: [pass] from rule [a/24]) reciever=shell.msen.com; client-ip=85.10.195.197; envelope-from=<vm-dev-return-630-lewis=mail.msen.com at discuss.squeakfoundation.org>; helo=box2.squeakfoundation.org;)
X-Milter: Spamilter (Reciever: shell.msen.com; Sender-ip: 85.10.195.197; Sender-helo: box2.squeakfoundation.org;)
Status: RO
Content-Length: 1063
Lines: 28


On 21-Apr-06, at 8:39 PM, Ian Piumarta wrote:

> David,
>
>> p.s. I think this power of two business is a bit over-hyped. The
>> correct machine word size is and always has been 24 bits.
>
> 24?  24???!!!  What kind of number is that?!?  You can't even fit  
> an entire file name into it (sixbit uppercase ASCII, naturally)!
>
> Noooo....  You need 36 bits, my man.  Split down the middle for  
> those ordinary everyday tasks where 18 bit halfwords will do just  
> fine, and in only half the resources too.

Fie on that nonsense. What y'need is 33bits. 32 for a decent size  
integer (who*needs* 64 bit values, anyway) and one bit for a tag.  
Then you  need an asynchronous ARM-like architecture with the TLC  
(sorta WCS) and floating pint hardware connected to fullspeed mram so  
that there is no need for data cache. Then you have a few thousand of  
those in a single machine. Easy. Now gimme One Billion Euros to  
implement it.

tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
State-of-the-art: What we could do with enough money.




More information about the Squeak-dev mailing list