From: Tim Hammerquist <tim@...>
Date: 2003-09-20T08:20:54+09:00
Subject: Re: Trouble with binary files?

<agemoagemo@yahoo.com> graced us by uttering:
> <werntges@informatik.fh-wiesbaden.de> wrote:
> > > files. Is there something else I have to do to be able to
> > > work with binary data?
>
> > Did you try IO#binmode ? Unix guys (like me) typically miss
> > that when working in a Windows environment.
>
> Nope, hadn't seen that. And that fixes it.
>
> I don't particularly understand *why* though. Those first 160
> bytes were being read properly, what stops it from getting the
> rest? Why 160? Is it going to suddenly require something else
> to be done when I go over to the 1.73mb file that the program
> is designed to process? (I wouldn't *think* so, but then I
> didn't expect this 160 thing either...)

You misunderstand the 160-byte barrier as being related to Ruby.
It's a Win32/DOS issue.

<story-mode>

Way back in PC-/MS-DOS days, it was decided that the
non-printable ASCII-26 (^Z) character would mark the end of a
textmode file.  The difference between textmode and binmode of a
DOS file is important, though it need not ever have become an
issue.

The requirement for ^Z to terminate a textfile has since been
changed.  However, (for backward compatibility?) when the ^Z *is*
encountered in a textmode file, DOS (and subsequently, Windows)
still set the EOF flag and stop reading.

</story-mode>

This becomes more of an issue when the default file open mode for
DOS/Win is in text mode, creating the need for a completely new
function call almost exclusively for DOS/Win platforms; in this
case, binmode(), which explicitly sets the file read mode to
binary, preventing the OS from stopping at the first ^Z (and from
changing line endings, blah, blah...).

As someone else mentioned above, this isn't an issue on Unix or
many other systems, since EOF on these OSes isn't determined by
file contents.  I'm not if it's an issue for Macs, as they also
historically use different line endings.  This also may have
changed with OS X; anyone know?

The moral of the story is:

    Always call fh.binmode() before reading
    any non-text file on non-Unix platforms.

HTH,
Tim Hammerquist
-- 
scanf() is evil.