[squeak-dev] filesize reporting 0 for very large files

Eliot Miranda eliot.miranda at gmail.com
Mon Mar 19 01:55:10 UTC 2012


Hi David,

On Sun, Mar 18, 2012 at 6:30 PM, David T. Lewis <lewis at mail.msen.com> wrote:

> On a Linux system, the data types that represent file positions are 32 bits
> in size when the program is compiled in 32-bit mode (using the -m32
> compiler
> option), and they are 64 bits in size when the program is compiled in
> 64-bit
> mode. The relevant data types are off_t and size_t, and if you compile the
> following in 32-bit mode (-m32) and compare the same program compiled in
> 64-bit mode, you can see the difference:
>
>  #include <stdio.h>
>  #include <sys/types.h>
>  main() {
>    printf("off_t is %d\n", sizeof(off_t));
>    printf("size_t is %d\n", sizeof(size_t));
>  }
>

Not quite.  One can modify this by defining something like
_LARGEFILE64_SOURCE at compile time.  e.g.
see _LARGEFILE_SOURCE _LARGEFILE64_SOURCE & _FILE_OFFSET_BITS in
http://www.delorie.com/gnu/docs/glibc/libc_13.html.  I'll check that the
appropriate one is defined when building Cog asap.



>
> The VM is a program like any other, so when compiled as a 32-bit
> application,
> file operations will fail if sizes and offsets exceed the range of an
> off_t data type.
>
> If you are dealing with very large files, you will need to consider using
> a VM compiled in 64-bit mode, or you will need to find some mechanism to
> limit data file size.
>
> Cog is currently limited to 32-bit mode, so changing VMs may not be
> acceptable if performance is a key concern. An interpreter VM compiled
> in 64-bit mode is entirely suitable for server applications, except that
> FFI and certain plugins will not be available.
>
> Most users do not require large address spaces or data files, and the VMs
> in general circulation are compiled in 32-bit mode in order to provide
> support for a wide range of plugins.
>
> Dave
>
>
> Sun, Mar 18, 2012 at 11:10:17AM -0500, Chris Muller wrote:
> > Does a 32-bit VM mean that no individual primitives can have an
> > argument or return value greater than 32 bits?  File-size is something
> > that, in 2012 with HD video recording devices, can very easily exceed
> > 32-bits so it would seem to be overkill for ALL objects to have to be
> > larger than 32 bits (as in a 64-bit VM) just for this one primitive..
> >
> >
> > On Thu, Mar 15, 2012 at 11:45 PM, David T. Lewis <lewis at mail.msen.com>
> wrote:
> > > On Thu, Mar 15, 2012 at 10:35:47PM -0500, Chris Muller wrote:
> > >> I have a 3.2GB file, but Squeak reports its #fileSize as 0 (in the
> > >> DirectoryEntry). ?It seems to occur for any file that large.
> > >>
> > >> Is it an overflow condition in the VM? ?Is it possible to fix?
> > >
> > > It works fine with an interpreter VM compiled in 64-bit mode. I
> > > cannot look into it in detail now, but an educated guess is that
> > > it relates to the definition of size_t for 32-bit programs, and
> > > that for the kind of work you are doing here a 64-bit VM may be
> > > in order.
> > >
> > > Dave
> > >
>
>


-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20120318/3f8a7c2d/attachment.htm


More information about the Squeak-dev mailing list