From: Aredridel <aredridel@...>
Date: 2003-04-19T01:30:14+09:00
Subject: File Type problems / Re: How to test for text file

Peter B. Ensch wrote:
> Ah! Not having worked with Unicode before, I had not considered this.
> Does your answer assume that I need to define 'text file' and then
> open/read the file to see if it meets the definition? In other words,
> it is not possible to do something like:
> if text file
>   open and process
> else 
>   do not open
> end

That's what perl does as well, internally -- it opens and checks the (I
believe) first line for characters that indicate one way or the other
(NUL is a good indication of binary, CRLF is a good indication it's
text, under Windows.  Under Unix, and dealing with other character sets,
things get way hairier)

I've never found a good way to detect file types easily -- there's the
"file" command under many unices that looks in a database of magic
numbers to guess file formats -- GIF files start with GIF8?a, where ? is
7 or 9.  JPEG files start with JFIF, windows executables with MZ and
linux and many other unices use ELF.  Windows NT and later stamp unicode
text files with an "ordering sequence", I can't remember which, but if
it's interpreted with the bytes swapped, it's invalid unicode, but if
correct, is a two byte "NUL" sequence, telling windows which byte order
the file is in.  If the file command doesn't see anything that indicates
a certain type, it just guesses similarly to perl and says "Binary data"
or "text file".

Windows and DOS try to solve this with file extensions, of course, and
Macs with creator codes, and nowadays, being unix-based, it's starting
to use some combination of MacOS magic and file extensions and guessing,
which starts to get confusing.

Unix traditionally avoids the issue by making the user decide what to
open, or just throwing errors when it gets an ugly file, like a word
processor opening a jpeg.  Such fun.

Aredridel