[Vm-dev] 64bit: "This interpreter (vers. 68000) cannot read image file (vers. 274877913446)."

[David T. Lewis]

On Wed, Oct 17, 2007 at 09:26:35AM +0200, Andrew Gaylard wrote:
>  
> On 10/17/07, David T. Lewis <lewis at mail.msen.com> wrote:[snip]
> 
> > - What is a 64 bit VM?
> > A 64 bit image is an image in which the object memory uses 64 bit word
> > size for object pointers. Squeak now supports a 64 bit image format that
> > is sufficient to produce a working system, but which is intentionally
> > simple, and is expected to be modified and extended to take advantage of
> > additional 64 bit capabilities in the future.
> 
> 
> Considering the answer you gave,  did you mean this question to be
> "What is a 64-bit image?"?

Yes, that's what I meant, sorry.

> Here's a pass at answering the "what is a 64-bit VM?" question:
> A 64-bit VM is one which is compiled with the LP64 or ILP64 data
> model.  This means, in C terms, that pointers and longs are 64 bits
> wide.

Good.

> - Can I run a 32- or 64-bit VM on my computer?
> Some current architectures (such as the x86-64 and the UltraSPARC)
> can run 32-bit as well as 64-bit applications; these are known as
> "bi-arch".  However, some (such as the Alpha) can only run 64-bit
> applications.  For bi-arch systems, you can choose whether to run
> a 32-bit or 64-bit VM.  For 64-bit-only systems, you can only run a
> 64-bit VM, since there's no way of compiling a 32-bit application.

Good.

> - How does a 32-bit VM manage to run a 64-bit image if pointers are 32-bits?
> (I'm guessing here!)
> It relies on the image size being smaller than 4GB. Code in the VM
> converts the 64-bit object pointers to 32-bit ones that the VM can use.

Object pointers within the object memory are not pointers in the C
sense of the word. The VM needs to be able to convert the object pointers
into C pointers, and this can be done on either a 32 bit host or a 64
bit host.  The only caveat would be that if a 64 bit image grew to a
size large enough to use object pointers larger than the 32 bit limit
(i.e. an image approaching 4GB in size), then a 64 bit VM would be required.

> - What sizes and alignment does the new 64-bit image format use for
> pointers and integers?
> (Again, I'm guessing here)
> In the 64-bit image format, object pointers are 64-bits wide, allowing for
> memory up to 2^64 bytes to be directly addressable.  They are aligned
> on 8-byte boundaries.
> Integers are still implemented as tagged 31-bit values.  They are aligned
> to 4-byte boundaries.
> These alignments were chosen as most 64-bit CPUs require them.

The object header and pointer formats for both 32 bit and 64 bit
images are documented in the class comment of ObjectMemory (in the
VMMaker package). The conversions to and from host data types are
done in platforms/Cross/vm/sqMemoryAccess.h using either macros or
inline functions. The actual conversions vary from host to host, and
are controlled by macros such as SQ_HOST64 and SQ_IMAGE32 that must
be for that host. In the case of a Unix VM, the configure utility
is used to specify the characteristics of the host platform.

The word size to be used in the object memory is specified by
the SQ_VI_BYTES_PER_WORD macro on src/vm/interp.h. This file is
created in the VMMaker code generation process, and the value of
SQ_VI_BYTES_PER_WORD is determined by the "64 bit VM?" check box
in the VMMaker tool.

In summary, the object word format is described in class ObjectMemory,
the host data type conversions are specified in sqMemoryAccess.h,
and the image word size is specified in interp.h.

> - how do I tell if a given image file is 32-bit or 64-bit?

The first four bytes of the image in the image file are a "magic"
value that indicates the image word size. This is specified in
Interpreter>>imageFormatVersion. For most images, this
will be the first four bytes of the image file, although in some
cases the image data may be offset by 512 bytes in order to
permit an image file to be treated as an executable program on
Unix platforms (see http://en.wikipedia.org/wiki/Shebang_(Unix)).

Dave