From: Hal Fulton <hal9000@...>
Date: 2003-09-20T08:41:25+09:00
Subject: Re: Trouble with binary files?

Tim Hammerquist wrote:
> You misunderstand the 160-byte barrier as being related to Ruby.
> It's a Win32/DOS issue.
> 
> <story-mode>
> 
> Way back in PC-/MS-DOS days, it was decided that the
> non-printable ASCII-26 (^Z) character would mark the end of a
> textmode file.  The difference between textmode and binmode of a
> DOS file is important, though it need not ever have become an
> issue.
> 
> The requirement for ^Z to terminate a textfile has since been
> changed.  However, (for backward compatibility?) when the ^Z *is*
> encountered in a textmode file, DOS (and subsequently, Windows)
> still set the EOF flag and stop reading.
> 
> </story-mode>
> 
> This becomes more of an issue when the default file open mode for
> DOS/Win is in text mode, creating the need for a completely new
> function call almost exclusively for DOS/Win platforms; in this
> case, binmode(), which explicitly sets the file read mode to
> binary, preventing the OS from stopping at the first ^Z (and from
> changing line endings, blah, blah...).
> 
> As someone else mentioned above, this isn't an issue on Unix or
> many other systems, since EOF on these OSes isn't determined by
> file contents.  I'm not if it's an issue for Macs, as they also
> historically use different line endings.  This also may have
> changed with OS X; anyone know?
> 
> The moral of the story is:
> 
>     Always call fh.binmode() before reading
>     any non-text file on non-Unix platforms.

True, but let's be fair.

MSDOS stole many things from Unix, such as the notion of a
hierarchical directory structure and the use of < > | at the
shell level. (Many things were incompletely stolen, unfortunately.)

The binmode/textmode distinction came from Unix. At that time
Unix had an EOF character of control-D (which explains the ^D we
still type occasionally at the terminal).

So historically Unix's behavior with respect to ^D was the same as
DOS's with respect to ^Z. But Unix/Linux moved beyond that, and
DOS/Windows never did.

Hal