From: jonathan@... (Jonathan Baker)
Date: 2002-03-09T18:33:33+09:00
Subject: Re: Spooky backtick bug w/large file support

By "32-bit platform", I meant that the pointer size is 32 bits, not
that it doesn't have large file support.  The issue with the read
command below (not likely to be an everyday occurrence, admittedly) is
that you've tried to slurp up a file bigger than 2^32 bytes into
memory all at once, which you can't do -- even though with large file
support of course you can seek around anywhere in the file and read
chunks up to 2^32.  So I was arguing against changing the type to
off_t in io_fread, because you're not really allowed to use something
bigger than a long anyway.

You're right that open raises EBIG without LFS, and you do need NFS
v3.  I also noticed that without LFS, FileTest.exists? silently
returns false on large files.

Here's a simple way to make a nice large file for testing the patch.

  --Jonathan


#include <stdio.h>

/* Ok so this is not the fastest way to make a large file.  :)
 * But it makes one where each 10-byte line contains the file 
 * offset at the start of the line divided by 10, so we can easily
 * seek around and see if we're in the right place, and we're fine
 * with 32-bit ints here. */

#define N_LINES  3 * 1024 * 1024 * 103   /* => ~3 GB file */

int
main(int argc, char **argv)
{
    unsigned long i;
    for (i = 0; i < N_LINES; i++) printf("%09ld\n", i);
    return 0;
}




"Oliver M. Bolzer" <oliver@fakeroot.net> wrote in message news:<a6brhp$epb$1@minotaurus.cip.informatik.uni-muenchen.de>...
> Jonathan Baker <jonathan@alumni.caltech.edu> wrote:
> 
> > This brings up the question of what the behavior should be when one
> > (accidentally) does this on a 32-bit platform:
> > 
> >     s = File.open("some_large_file").read
> 
> At least when I checked my version of the (more-or-less) same patch, the 
> underlying C functions returned errors when opening large files on systems
> (or Rubys) that don't support them, that where correctly transformed in to 
> the  Ruby Exception .Errno::EBIG Or if you wrote a file beyond 2^31, the 
> program would receive a SIGXFSZ and terminate (unless the signal is 
> trapped). The same will happen even on a LFS-supporting system and Ruby if 
> you access a large file over NFSv2 because NFSv2 doesn't support large 
> files. I almost went insane because my test would run in /tmp but not 
> inside my $HOME. Using NFSv3 solves this.
> 
> I really need to dig out my Rubicon patch to test LFS but the box they were 
> on is currently down and I couldn't find anybody to remote control across 
> the Eurasian continent to switch it on. Everybody on vacation (including me 
> :-)
> 
> My test was done on a Linux 2.4 system with glibc 2.2