[squeak-dev] filesize reporting 0 for very large files

David T. Lewis lewis at mail.msen.com
Mon Mar 19 01:30:45 UTC 2012


On a Linux system, the data types that represent file positions are 32 bits
in size when the program is compiled in 32-bit mode (using the -m32 compiler
option), and they are 64 bits in size when the program is compiled in 64-bit
mode. The relevant data types are off_t and size_t, and if you compile the
following in 32-bit mode (-m32) and compare the same program compiled in
64-bit mode, you can see the difference:

  #include <stdio.h>
  #include <sys/types.h>
  main() {
    printf("off_t is %d\n", sizeof(off_t));
    printf("size_t is %d\n", sizeof(size_t));
  }

The VM is a program like any other, so when compiled as a 32-bit application,
file operations will fail if sizes and offsets exceed the range of an
off_t data type.

If you are dealing with very large files, you will need to consider using
a VM compiled in 64-bit mode, or you will need to find some mechanism to
limit data file size.

Cog is currently limited to 32-bit mode, so changing VMs may not be
acceptable if performance is a key concern. An interpreter VM compiled
in 64-bit mode is entirely suitable for server applications, except that
FFI and certain plugins will not be available.

Most users do not require large address spaces or data files, and the VMs
in general circulation are compiled in 32-bit mode in order to provide
support for a wide range of plugins.

Dave


Sun, Mar 18, 2012 at 11:10:17AM -0500, Chris Muller wrote:
> Does a 32-bit VM mean that no individual primitives can have an
> argument or return value greater than 32 bits?  File-size is something
> that, in 2012 with HD video recording devices, can very easily exceed
> 32-bits so it would seem to be overkill for ALL objects to have to be
> larger than 32 bits (as in a 64-bit VM) just for this one primitive..
> 
> 
> On Thu, Mar 15, 2012 at 11:45 PM, David T. Lewis <lewis at mail.msen.com> wrote:
> > On Thu, Mar 15, 2012 at 10:35:47PM -0500, Chris Muller wrote:
> >> I have a 3.2GB file, but Squeak reports its #fileSize as 0 (in the
> >> DirectoryEntry). ?It seems to occur for any file that large.
> >>
> >> Is it an overflow condition in the VM? ?Is it possible to fix?
> >
> > It works fine with an interpreter VM compiled in 64-bit mode. I
> > cannot look into it in detail now, but an educated guess is that
> > it relates to the definition of size_t for 32-bit programs, and
> > that for the kind of work you are doing here a 64-bit VM may be
> > in order.
> >
> > Dave
> >


More information about the Squeak-dev mailing list